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Highlights 


Why  GAO  Did  This  Study 


What  GAO  Found 


The  Joint  Strike  Fighter  (JSF) 
program  seeks  to  produce  and  field 
three  aircraft  variants  for  the  Air 
Force,  Navy,  Marine  Corps,  and 
eight  international  partners.  The 
estimated  total  investment  for  JSF 
now  approaches  $1  trillion  to 
acquire  and  maintain  2,458  aircraft. 

Under  congressional  mandate, 

GAO  has  annually  reviewed  the  JSF 
program  since  2005.  GAO’s  prior 
reviews  have  identified  a  number 
of  issues  and  recommended  actions 
for  reducing  risks  and  improving 
the  program’s  outcomes. 

This  report,  the  fourth  under  the 
mandate,  focuses  on  the  program’s 
progress  in  meeting  cost,  schedule, 
and  performance  goals;  plans  and 
risks  in  development  and  test 
activities;  the  program’s  cost¬ 
estimating  methods;  and  future 
challenges  facing  the  program. 

To  conduct  its  work,  GAO 
identified  changes  in  cost  and 
schedule  from  prior  years  and  their 
causes,  evaluated  development 
progress  and  plans,  assessed  cost¬ 
estimating  methodologies  against 
best  practices,  and  analyzed  future 
budget  requirements. 


What  GAO  Recommends 


GAO  recommends  that  DOD  revisit 
and,  if  appropriate,  revise  the  Mid- 
Course  Risk  Reduction  Plan  to 
address  concerns  about  testing,  use 
of  management  reserves,  and 
manufacturing.  GAO  also 
recommends  action  to  improve  the 
reliability  and  fidelity  of  the  JSF 
cost  estimate.  DOD  substantially 
agreed. 


To  view  the  full  product,  including  the  scope 
and  methodology,  click  on  GAO-08-388. 

For  more  information,  contact  Michael  J. 
Sullivan  (202)  512-4841  or 
sullivanm@gao.gov. 


Since  last  year’s  report,  the  JSF  program  office  estimates  that  total  acquisition 
costs  increased  by  more  than  $23  billion,  primarily  because  of  higher 
estimated  procurement  costs.  The  JSF  development  cost  estimate  stayed 
about  the  same.  Development  costs  were  held  constant  by  reducing 
requirements,  eliminating  the  alternate  engine  program,  and  spending 
management  reserve  faster  than  budgeted.  Facing  a  probable  contract  cost 
overrun,  DOD  implemented  a  Mid-Course  Risk  Reduction  Plan  to  replenish 
management  reserves  from  about  $400  million  to  about  $1  billion  by  reducing 
test  resources.  Progress  has  been  reported  in  several  important  areas, 
including  partner  agreements,  first  flights  of  a  JSF  prototype  and  test  bed,  and 
a  more  realistic  procurement  schedule. 

The  midcourse  plan  carries  the  risk  of  design  and  performance  problems  not 
being  discovered  until  late  in  the  operational  testing  and  production  phases, 
when  it  is  significantly  more  costly  to  address  such  problems.  The  plan  also 
fails  to  address  the  production  and  schedule  concerns  that  depleted 
management  reserves.  Cost  and  schedule  pressures  are  mounting.  Two-thirds 
of  budgeted  funding  for  JSF  development  has  been  spent,  but  only  about  one- 
half  of  the  work  has  been  completed.  The  contractor  is  on  its  third,  soon  to  be 
fourth,  manufacturing  schedule,  but  test  aircraft  in  manufacturing  are  still 
behind,  the  continuing  impacts  of  late  designs,  delayed  delivery  of  parts,  and 
manufacturing  inefficiencies. 

We  believe  that  JSF  costs  will  likely  be  much  higher  than  reported.  The 
estimates  do  not  include  all  costs,  including  about  $6.8  billion  for  the  alternate 
engine  program.  In  addition,  some  assumptions  are  overly  optimistic  and  not 
well  documented.  Three  independent  defense  offices  separately  concluded 
that  program  cost  estimates  are  understated  by  as  much  as  $38  billion  and 
that  the  development  schedule  is  likely  to  slip  from  12  to  27  months. 
Discrepancies  in  cost  estimates  add  to  program  risks  and  hinder 
congressional  oversight.  Even  so,  DOD  does  not  plan  for  another  fully 
documented,  independent  total  program  life-cycle  cost  estimate  until  2013. 

As  JSF  finalizes  the  three  designs,  matures  manufacturing  processes, 
conducts  flight  tests,  and  ramps  up  production,  it  faces  significant  challenges. 
JSF’s  goal — to  develop  and  field  an  affordable,  highly  common  family  of  strike 
aircraft — is  threatened  by  rising  unit  procurement  prices  and  lower 
commonality  than  expected.  The  program  also  makes  unprecedented  funding 
demands — an  average  of  $11  billion  annually  for  two  decades — and  must 
compete  with  other  defense  and  nondefense  priorities  for  the  shrinking 
federal  discretionary  dollar.  Further,  expected  cost  per  flight  hour  now 
exceeds  that  of  the  F-16  legacy  fighter,  one  of  the  aircraft  it  is  intended  to 
replace.  With  almost  90  percent  (in  terms  of  dollars)  of  the  acquisition 
program  still  ahead,  it  is  important  to  address  these  challenges,  effectively 
manage  future  risks,  and  move  forward  with  a  successful  program  that  meets 
our  and  our  allies’  needs. 
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United  States  Government  Accountability  Office 
Washington,  DC  20548 


March  11,  2008 
Congressional  Committees 

The  Joint  Strike  Fighter  (JSF)  is  Department  of  Defense’s  (DOD)  most 
complex  and  ambitious  aircraft  acquisition,  seeking  to  simultaneously 
produce  and  field  three  aircraft  variants  for  the  Air  Force,  Navy,  Marine 
Corps,  and  eight  international  partners.  For  the  United  States,  the  JSF  will 
need  a  joint,  long-term  commitment  to  very  large  annual  funding 
requirements  and  a  total  investment  now  approaching  $1  trillion  dollars — 
$300  billion  to  acquire  2,458  aircraft  and  $650  billion  in  life-cycle  operation 
and  support  costs,  according  to  official  program  estimates.  The  JSF  is 
critical  to  our  nation’s  plans  for  recapitalizing  tactical  aircraft  and  just  as 
important  to  our  allies. 

The  Ronald  W.  Reagan  National  Defense  Authorization  Act  for  Fiscal  Year 
2005  requires  GAO  to  review  the  JSF  program  annually  for  five  years.1 
Previous  reports  identified  opportunities  for  the  program  to  reduce  risks 
and  improve  the  chance  for  more  successful  outcomes.  We  have 
expressed  concern  about  the  substantial  overlap  of  development,  test,  and 
production  activities  and  recommended  a  more  evolutionary  and 
knowledge-based  acquisition  strategy  with  limited  investment  in 
production  aircraft  until  each  variant  demonstrates  required  capabilities  in 
flight  testing.2  This  is  the  fourth  report  under  the  mandate  in  which  we  (1) 
determine  the  JSF  program’s  progress  in  meeting  cost,  schedule,  and 
performance  goals;  (2)  assess  plans  and  risks  in  development  and  test 
activities;  (3)  evaluate  program  office  cost-estimating  methodology;  and 
(4)  identify  future  challenges  facing  the  program. 

The  act  also  requires  us  to  certify  whether  we  had  access  to  sufficient 
information  to  make  informed  judgments  on  the  matters  contained  in  our 
report.  While  we  were  provided  sufficient  information  to  make  the 


1  Pub.  L.  No.  108-375,  §  213  (2004). 

2  GAO,  Joint  Strike  Fighter:  Progress  Made  and  Challenges  Remain,  GAO-07-360 
(Washington,  D.C.:  Mar.  15,  2007);  Joint  Strike  Fighter:  DOD  Plans  to  Enter  Production 
before  Testing  Demonstrates  Acceptable  Performance,  GAO-06-356  (Washington,  D.C.: 
Mar.  15,  2006);  and  Tactical  Aircraft:  Opportunity  to  Reduce  Risks  in  the  Joint  Strike 
Fighter  Program,  with  Different  Acquisition  Strategy,  GAO-05-271  (Washington,  D.C.: 
Mar.  15,  2005). 
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assessments  contained  in  this  report,  a  continuing  concern  has  been  the 
currency  of  cost  and  schedule  data.  The  data  we  are  reporting  here  are 
from  the  JSF  Selected  Acquisition  Report  (SAR)  dated  December  2006  that 
was  released  to  Congress  and  us  in  April  2007.  That  SAR  reflects  the 
program  position  at  the  time  of  the  submission  of  the  fiscal  year  2008 
President’s  Budget.  We  were  not  able  to  review  the  program  office’s 
updated  estimates  associated  with  the  fiscal  year  2009  budget,  which  will 
be  reported  in  the  new  December  2007  SAR  to  be  released  in  April  2008. 
The  program  office  declined  to  provide  updated  costs,  stating  that  those 
figures  are  sensitive  because  the  new  budget  request  had  not  been 
finalized  at  the  time  of  our  review.  Every  year,  this  timing  disconnect 
results  in  us  reporting  soon-to-be-outdated  cost  and  schedule  data.  For 
example,  shortly  after  our  last  report  was  issued  on  March  15,  2007,  DOD 
released  new  cost  estimates  that  disclosed  an  increase  of  more  than  $23 
billion  in  JSF  program  costs.  On  the  basis  of  the  evidence  we  do  have  and 
our  analysis,  we  fully  expect  future  cost  estimates  to  be  substantially 
higher  than  the  program  estimates  in  this  report. 

To  conduct  this  work,  we  tracked  and  compared  current  cost  and 
schedule  estimates  with  those  of  prior  years,  identified  major  changes,  and 
determined  causes.  We  obtained  earned  value  data,  contractor  workload 
statistics,  performance  indicators,  and  manufacturing  results.  We 
assessed  the  program  office’s  cost  estimating  methodologies  against  best 
practices  prescribed  in  GAO’s  Cost  Assessment  Guide.  We  discussed 
results  to  date,  plans,  and  future  challenges  with  DOD  and  contractor 
officials.  We  conducted  this  performance  audit  from  June  2007  to  March 
2008  in  accordance  with  generally  accepted  government  auditing 
standards.  Those  standards  require  that  we  plan  and  perform  the  audit  to 
obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe 
that  the  evidence  obtained  provides  a  reasonable  basis  for  our  findings 
and  conclusions  based  on  our  audit  objectives. 


Results  in  Brief 


The  JSF  total  acquisition  cost  estimate  increased  by  more  than  $23  billion 
since  our  March  2007  report  due  to  changes  in  procurement  costs. 
Principal  driving  factors  were  (1)  increased  unit  costs  from  extending  the 
procurement  period  seven  years  at  lower  annual  rates  and  (2)  increased 
future  price  estimates  based  on  contractor  proposals  for  the  first 
production  lot.  The  official  cost  estimate  for  development  remained  about 
the  same  in  total  as  it  has  since  the  program  was  restructured  in  2004. 
However,  this  was  largely  achieved  by  reducing  requirements,  not  fully 
funding  the  alternate  engine  program  despite  congressional  interest  in  the 
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program,  and  spending  management  reserves  much  faster  than  budgeted. 
Facing  a  probable  contract  cost  overrun,  DOD  officials  decided  not  to 
request  additional  funding  and  time  for  development,  opting  instead  to 
reduce  test  resources  in  order  to  replenish  management  reserves  from 
$400  million  to  $1  billion.  During  the  last  year,  DOD  and  the  contractor 
also  reported  progress  in  several  important  areas,  including  international 
partner  agreements,  first  flights  of  a  JSF  prototype  and  test  bed,  and  a 
more  realistic  procurement  schedule. 

The  recent  decision  to  replenish  management  reserves  by  reducing  test 
resources,  known  as  the  Mid-Course  Risk  Reduction  Plan,  significantly 
increases  the  risks  of  not  completing  development  testing  on  time  and  not 
finding  and  fixing  design  and  performance  problems  until  late  into 
operational  testing  and  production,  when  it  is  more  expensive  and 
disruptive  to  do  so.  The  plan  also  does  not  directly  address  and  correct  the 
continuing  production  and  schedule  concerns  that  depleted  management 
reserves.  We  expect  program  development  and  procurement  costs  to 
increase  substantially  and  schedule  pressures  to  worsen  based  on 
performance  to  date  and  the  conditions  that  gave  rise  to  the  risk  reduction 
plan.  Two-thirds  of  budgeted  funding  for  the  JSF  has  been  spent  on  the 
prime  development  contract,  but  only  about  one-half  of  the  work  has  been 
completed.  The  contractor  has  extended  manufacturing  schedules  several 
times,  but  test  aircraft  delivery  dates  continue  to  slip.  The  flight  test 
program  has  barely  begun,  but  faces  substantial  risks  with  reduced  assets 
as  design  and  manufacturing  problems  continue  to  cause  delays  that 
further  compress  the  time  available  to  complete  development.  The 
Director,  Operational  Test  and  Evaluation,  and  several  other  prominent 
defense  offices  objected  to  the  midcourse  plan  as  too  risky  because  it  does 
not  provide  adequate  resources  for  development  testing  or  resolve 
systemic  problems  that  depleted  management  reserves.  We  agree. 

We  do  not  think  the  program  cost  estimate  is  reliable  when  judged  against 
cost  estimate  standards  used  throughout  the  federal  government  and 
industry.  Specifically,  the  program  cost  estimate  (1)  is  not  comprehensive 
because  it  does  not  include  all  applicable  costs,  including  $6.8  billion  for 
the  alternate  engine  program;  (2)  is  not  accurate  because  some  of  its 
assumptions  are  overly  optimistic  and  not  supportable — such  as  applying 
a  weight  growth  factor  only  half  as  large  as  historical  experience  on 
similar  aircraft — and  because  the  data  system  relied  upon  by  the  prime 
contractor  and  the  program  office  to  report  and  manage  JSF  costs  and 
schedule  is  deficient;  (3)  is  not  well  documented  in  that  it  does  not 
sufficiently  identify  to  cost  analysts  the  primary  methods,  calculations, 
results,  rationales  and  assumptions,  and  data  sources  used  to  generate 
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cost  estimates;  and  (4)  is  not  credible  according  to  three  independent 
defense  offices  who  all  conclude  that  program  cost  estimates  are 
understated  by  as  much  as  $38  billion  and  that  the  development  schedule 
is  likely  to  slip  from  12  to  27  months.  Despite  this  and  all  the  significant 
events  and  changes  that  have  occurred  in  the  6  years  since  the  start  of 
system  development,  DOD  does  not  intend  to  accomplish  another  fully 
documented,  independent  total  program  life-cycle  cost  estimate  for 
another  6  years. 

The  JSF  is  entering  its  most  challenging  phase  as  it  finalizes  three  designs, 
matures  manufacturing  processes,  conducts  flight  tests,  and  ramps  up 
production.  The  first  and  foremost  challenge  is  affordability.  From  its 
outset,  the  JSF  goal  was  to  develop  and  field  an  affordable,  highly 
common  family  of  strike  aircraft.  That  goal  is  threatened  by  rising  unit 
procurement  prices  and  somewhat  lower  commonality  than  expected, 
raising  concerns  that  the  United  States  and  its  allies  may  not  be  able  to 
buy  as  many  aircraft  as  currently  planned.  The  program  also  makes 
unprecedented  demands  for  funding  from  the  defense  budget — an  annual 
average  of  about  $11  billion  for  the  next  two  decades — and  must  compete 
with  other  defense  and  non-defense  priorities  for  the  shrinking  federal 
discretionary  dollar.  Further,  informed  by  more  knowledge  as  the  program 
progresses,  DOD  doubled  its  projection  of  JSF  life-cycle  operating  and 
support  costs  compared  to  last  year’s  estimate  and  its  expected  cost  per 
flight  hour  now  exceeds  that  of  the  F-16  legacy  fighter  it  is  intended  to 
replace.  With  almost  90  percent  (in  terms  of  dollars)  of  the  acquisition 
program  still  ahead,  it  is  important  to  address  these  challenges,  effectively 
manage  future  risks,  and  move  forward  with  a  successful  program  that 
meets  our  and  our  allies’  needs. 

Because  of  the  elevated  risks  and  valid  objections  raised  by  the  test 
community  and  other  DOD  offices,  we  recommend  that  DOD  revisit  and,  if 
appropriate,  revise  the  Mid-Course  Risk  Reduction  plan  recently  approved. 
DOD  should  specifically  address  concerns  about  constrained  testing 
capacity,  the  integration  of  flight  and  ground  tests,  depletion  of 
management  reserves,  slippage  in  the  manufacturing  schedule,  and 
progress  made  in  correcting  deficiencies  in  the  contractor’s  earned  value 
management  system,  and  to  examine  in  depth  the  alternatives  to  the 
current  plan  that  could  reduce  risks.  To  enhance  congressional  oversight 
and  provide  DOD  management  with  a  higher-fidelity  and  more 
comprehensive  cost  estimate,  we  also  make  several  recommendations  to 
improve  cost  estimates,  in  particular  that  DOD  accomplish  this  year  a  new 
total  program  life-cycle  cost  estimate,  validated  by  the  Cost  Analysis 
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Improvement  Group,  that  includes  risk  analysis  and  meets  DOD  policy 
requirements  for  major  system  cost  estimates  at  milestones. 

DOD  substantially  agreed  with  our  recommendations.  DOD  believes  that 
the  midcourse  plan  is  a  cost-effective  approach  with  a  manageable  level  of 
risk  that  will  be  monitored  and  revised  if  necessary.  Regarding  our  three 
recommendations  on  cost  estimating,  DOD  indicated  that  it  will  implement 
all  elements  except  the  risk  and  uncertainty  analysis  that  it  believes  is  not 
warranted.  We  think  that  risk  and  uncertainty  analysis  is  an  important  tool 
that  establishes  a  confidence  interval  for  a  range  of  possible  costs — as 
opposed  to  a  single-point  estimate — and  facilitates  good  management 
decisions  and  oversight.  Such  analysis  is  a  best  practice  in  our  Cost 
Assessment  Guide  and  we  note  that  OSD’s  Cost  Analysis  Improvement 
Group  supports  and  uses  this  cost-estimating  tool. 


Background 


The  JSF  program  goals  are  to  develop  and  field  an  affordable,  highly 
common  family  of  stealthy,  next-generation  strike  fighter  aircraft  for  the 
Navy,  Air  Force,  Marine  Corps,  and  U.S.  allies.  The  JSF  family  consists  of 
three  variants.  The  conventional  takeoff  and  landing  (CTOL)  variant  will 
primarily  be  an  air-to-ground  replacement  for  the  Air  Force’s  F-16  Falcon 
and  the  A-10  Warthog  aircraft,  and  will  complement  the  F-22A  Raptor.  The 
short  takeoff  and  vertical  landing  (STOVL)  variant  will  be  a  multirole 
strike  fighter  to  replace  the  Marine  Corps’  F/A-18C/D  and  AV-8B  Harrier 
aircraft.  The  carrier-suitable  (CV)  variant  will  provide  the  Navy  a  multi¬ 
role,  stealthy  strike  aircraft  to  complement  the  F/A-18E/F  Super  Hornet. 
DOD  is  planning  to  buy  a  total  of  2,458  JSFs.  The  F-35  JSF  was  christened 
Lightning  II  in  July  2006. 

Because  of  the  program’s  sheer  size  and  the  numbers  of  aircraft  it  will 
replace,  the  JSF  is  the  linchpin  of  DOD’s  long-term  plan  to  modernize 
tactical  air  forces.  It  is  DOD’s  largest  acquisition  program,  with  total  cost 
currently  estimated  at  $300  billion;  the  longest  in  planned  duration,  with 
procurement  projected  through  2034;  and  the  largest  cooperative 
international  development  program.3  Our  international  partners  are 
providing  about  $4.8  billion  toward  development,  and  foreign  firms  are 


3  The  international  partners  are  the  United  Kingdom,  Italy,  the  Netherlands,  Turkey, 
Canada,  Australia,  Denmark,  and  Norway.  These  nations  are  contributing  funds  for  system 
development  and  have  signed  agreements  to  procure  a  minimum  of  646  aircraft.  Israel  and 
Singapore  are  security  cooperation  participants,  and  several  other  nations  have  reportedly 
expressed  interest  in  acquiring  aircraft. 
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part  of  the  industrial  base  producing  aircraft.  They  are  expecting  to 
procure  a  minimum  of  646  CTOL  and  STOVL  JSFs.  DOD’s  funding 
requirements  for  the  JSF  assume  the  benefits  in  reduced  unit  costs  from 
these  purchases. 

Figure  1  shows  the  JSF’s  current  procurement  profile  for  U.S.  and 
international  partners.  Partner  purchases  begin  in  2009  and  reach  a 
maximum  of  95  per  year  in  fiscal  year  2016.  Total  expected  procurement  in 
that  peak  year,  including  U.S.  quantities,  is  225  aircraft. 


Figure  1:  JSF  Annual  Procurement  Plans  for  the  United  States  and  International  Partners 

Number  of  aircraft 
250 


Fiscal  year 


| _ |  U.S.  buys 

|  |  International  buys 

Source:  GAO  analysis  of  DOD  data. 


The  JSF  is  a  single-seat,  single-engine  aircraft,  designed  to  rapidly 
transition  between  air-to-ground  and  air-to-air  missions  while  still 
airborne.  To  achieve  its  mission,  JSF  will  incorporate  low-observable 
technologies,  defensive  avionics,  advanced  onboard  and  offboard  sensor 
fusion,  internal  and  external  weapons,  and  advanced  prognostic 
maintenance  capability.  According  to  DOD,  these  technologies  represent  a 
quantum  leap  over  legacy  tactical  aircraft  capabilities.  In  several  ways,  JSF 
development  is  also  more  complex  and  challenging  than  the  F-22A  Raptor 
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and  F/A-18E/F  Super  Hornet  programs,  the  other  two  contemporary 
aircraft  that  DOD  is  acquiring  with  JSF  to  recapitalize  tactical  air  forces. 
The  JSF  program  is  simultaneously  developing  several  airframes  and 
engines  for  multiple  customers  and  is  projected  to  have  significantly  more 
lines  of  operational  flight  plan  software  code  than  the  other  aircraft. 

The  JSF  program  began  in  November  1996  with  a  5-year  competition 
between  Lockheed  Martin  and  Boeing  to  determine  the  most  capable  and 
affordable  preliminary  aircraft  design.  Lockheed  Martin  won  the 
competition.  The  program  entered  system  development  and  demonstration 
in  October  2001.  At  that  time,  officials  planned  on  a  10 '/a  years 
development  period  costing  about  $34  billion  (amount  includes  costs  of 
about  $4  billion  incurred  before  system  development  start).  By  2003, 
system  integration  efforts  and  a  preliminary  design  review  revealed 
significant  airframe  weight  problems  that  affected  the  aircraft’s  ability  to 
meet  key  performance  requirements.  Weight  reduction  efforts  were 
ultimately  successful  but  added  substantially  to  program  cost  and 
schedule  estimates.  In  March  2004,  DOD  rebaselined  the  program  (2004 
Replan),  extending  development  by  18  months  and  adding  $7.5  billion  to 
development  costs.  Program  officials  also  delayed  the  critical  design 
reviews,  first  flights  of  development  aircraft,  and  the  low-rate  initial 
production  decision  to  allow  more  time  to  mitigate  risks  and  mature 
designs. 


Progress  Measured 
against  Cost, 
Schedule,  and 
Performance  Goals 
Was  Mixed  over  the 
Last  Year 


The  total  program  acquisition  cost  estimate  by  the  JSF  program  office  has 
increased  since  our  report  last  year,  primarily  due  to  higher  projected 
procurement  unit  prices.  The  reported  schedule  for  major  events  showed 
mostly  minor  slips.  Engineering  analyses  continue  to  show  performance 
requirements  are  met,  but  flight  and  ground  tests  planned  through  2013 
will  be  necessary  to  confirm  these  assessments.  DOD  and  the  contractor 
reported  progress  in  several  areas,  including  international  partner 
agreements,  first  flights  of  a  JSF  prototype  and  test  bed,  and  a  more 
realistic  procurement  schedule. 


The  Program  Cost 
Estimate  Increased,  while 
Schedule  and  Performance 
Estimates  Remained  about 
the  Same 


JSF  costs  increased  since  last  year.  Table  1  shows  the  evolution  of  cost, 
quantity,  and  delivery  estimates  from  the  initiation  of  system  development, 
through  the  2004  Replan,  to  the  latest  data  available.  It  demonstrates  the 
impacts  of  higher  procurement  costs  on  unit  costs  and  schedule  delays  on 
the  delivery  of  promised  capabilities  to  the  warfighters. 
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Table  1:  Changes  in  Reported  JSF  Program  Costs,  Quantities,  and  Deliveries 

October  2001 
(system 
development 
start) 

December  2003a 
(2004  Replan) 

December  2005a 

December  2006a 

(latest  available  data) 

Expected  quantities 

Development  quantities 

14 

14 

15 

15b 

Procurement  quantities  (U.S.  only) 

2,852 

2,443 

2,443 

2,443 

Total  quantities 

2,866 

2,457 

2,458 

2,458 

Cost  Estimates  (then-year  dollars  in  billions) 


Development 

$34.4 

$44.8 

$44.5 

$44.2 

Procurement 

196.6 

199.8 

231.7 

255.1 

Military  construction0 

2.0 

0.2 

0.2 

0.5 

Total  program  acquisition 

$233.0 

$244.8 

$276.5 

$299.8 

Unit  Cost  Estimates  (then-year  dollars  in  millions) 


Program  acquisition 

$81 

$100 

$112 

$122 

Average  procurement 

69 

82 

95 

104 

Estimated  delivery  dates 

First  operational  aircraft  delivery 

2008 

2009 

2009 

2010 

Initial  operational  capability 

2010-2012 

2012-2013 

2012-2013 

2012-2015 

Source:  GAO  analysis  of  DOD  data. 

aData  are  from  the  annual  Selected  Acquisition  Reports  that  are  dated  in  December  but  not  officially 
released  until  March  or  April  of  the  following  year.  The  December  2003  data  reflect  the  2004  Replan. 
The  December  2006  data  are  the  latest  information  on  total  program  costs  made  available  to  us  by 
DOD. 

bA  subsequent  decision  by  DOD  in  September  2007  has  reduced  development  test  aircraft  by  2  to  13. 

'Military  construction  costs  have  not  been  fully  established  and  the  reporting  basis  changed  overtime 
in  these  DOD  reports.  The  amount  shown  for  December  2006  represents  costs  currently  in  the  2008 
future  years  defense  plan. 


The  current  estimate  for  procurement  costs,  dated  December  2006,  shows 
an  increase  of  $23.4  billion  (plus  10  percent)  from  the  estimate  of  a  year 
earlier  and  a  total  of  $55.3  billion  more  (plus  28  percent)  since  2004.4 


4  To  eliminate  the  effects  of  inflation,  these  procurement  cost  increases  expressed  in  base 
year  fiscal  year  2002  dollars  are  $7.8  billion  (plus  5  percent)  and  $19.6  billion  (plus  13 
percent),  respectively. 
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Procurement  cost  increases  were  primarily  due  to  (1)  extending  the 
procurement  period  seven  years  at  lower  annual  rates,  (2)  increased  future 
price  estimates  based  on  contractor  proposals  for  the  first  production  lot, 
(3)  airframe  material  cost  increases,  and  (4)  increases  resulting  from 
design  maturation.  Offsetting  a  portion  of  the  procurement  cost  increases 
were  lower  estimates  for  labor  rates  and  subcontractor  costs. 

The  official  development  cost  estimate  has  remained  relatively  constant 
since  the  2004  Replan.  However,  there  were  significant  changes  in  scope 
and  planned  use  of  funds  in  order  to  maintain  that  estimate  as  officials 
reduced  requirements,  did  not  include  full  funding  for  the  alternate  engine 
program  despite  congressional  interest  in  the  program,5  and  spent 
management  reserves  much  faster  than  budgeted.  Management  reserves 
are  a  pool  of  money  set  aside — in  this  case  about  10  percent  of  the 
development  contract  value  remaining — to  handle  unanticipated  changes 
and  other  risks  encountered  as  a  development  program  proceeds.  Weight 
growth  early  in  development  and  subsequent  problems  resulting  from  late 
aircraft  design  changes  and  subsequent  manufacturing  inefficiencies 
depleted  reserve  funds  to  an  untenable  level  by  2007.  The  program  faced  a 
probable  contract  overrun.  DOD  officials  opted  not  to  request  additional 
funding  and  time  to  complete  development  and  instead  adopted  a 
controversial  plan  that  reduced  budgeted  funds  for  development  test 
aircraft  and  flight  plans  in  order  to  replenish  management  reserves  from 
$400  million  to  about  $1  billion,  an  amount  deemed  prudent  to  complete 
the  development  phase  on  time.  This  plan,  known  as  the  Mid-Course  Risk 
Reduction  plan,  is  discussed  in  more  detail  later  in  this  report. 

Reported  schedule  slips  for  key  events  since  last  year’s  report  were  minor 
for  the  most  part,  but  schedules  could  worsen  considerably  if  the  delays  in 
maturing  the  aircraft  and  engine  designs  and  manufacturing  test  aircraft 
continue  to  push  work  effort  into  later  years.  This  would  further  compress 
the  time  available  to  complete  development  and  test  efforts,  affecting  the 
scheduled  start  of  initial  operational  test  and  evaluation  and  the  full-rate 
production  decision,  and  increasing  the  risk  of  further  delivery  delays.  The 
CV’s  critical  design  review,  the  last  of  three  design  reviews  for  the 
program,  occurred  in  June  2007,  seven  months  later  than  had  been 
expected.  The  initial  operational  capability  date  for  this  variant  was 


5  Congress  has  subsequently  required  that  DOD  obligate  and  expend  sufficient  annual 
amounts  for  the  continued  development  and  procurement  of  the  alternate  engine  program. 
National  Defense  Authorization  Act  for  Fiscal  Year  2008,  Pub.  L.  No.  110-181,  §  213  (2008). 
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pushed  out  two  years  to  March  2015,  to  provide  more  time  to  mature 
design  and  test  this  variant  in  the  demanding  carrier  environment.  The 
carrier  variant  is  the  least  developed  of  the  three,  incorporates  larger 
wings,  is  heavier,  and  has  different  speed  and  range  performance 
requirements  than  the  other  two  variants. 

On  the  basis  of  engineering  analyses  and  computer  modeling,  the  JSF 
program  projects  that  the  aircraft  design  will  meet  seven  of  the  eight  key 
performance  parameters  by  the  end  of  development.  The  aircraft  is 
currently  not  meeting  the  interoperability  parameter,  but  this  depends  on 
capabilities  being  developed  outside  the  JSF  program.  Key  performance 
parameters  will  be  verified  during  ground  and  flight  testing  from  2010  to 
2013. 


Progress  Was  Made  This  DOD  and  the  contractor  made  solid  progress  this  year  in  several  areas  that 
Year  in  Several  Important  could  establish  a  foundation  to  spur  future  successes.  With  almost  90 

^.eas  percent  (in  terms  of  dollars)  of  the  acquisition  program  still  ahead,  these 

and  other  improvements  could  be  leveraged  to  help  better  meet  cost, 

schedule  and  performance  goals. 

•  In  February  2007,  the  United  States  and  eight  international  partners 
signed  the  Production,  Sustainment,  and  Follow-on  Development 
Memorandum  of  Understanding,  committing  to  purchase  aircraft  and 
continuing  joint  development  activities. 

•  DOD  reduced  near-term  procurement  quantities  and  the  rate  of  ramp- 
up  to  full  rate  production.  These  actions  somewhat  lessened  the 
concurrency  of  development  and  production  we  have  previously  cited 
and  make  for  a  more  achievable  schedule. 

•  The  prime  contractor  and  major  subcontractors  continued  to 
implement  advanced  design  and  development  techniques  and  utilize 
extensive  computer  modeling  and  simulation  in  innovative  ways  for 
design,  test,  and  integration  activities. 

•  DOD  and  contractor  officials  also  made  good  progress  toward  refining 
system  capabilities,  including  establishing  mission  software 
requirements,  with  the  goal  of  improving  future  program  executability 
while  still  meeting  warfighter  requirements. 

•  First  flights  of  the  prototype  test  aircraft  and  a  flying  test  bed  occurred 
in  fiscal  year  2007.  Both  are  viewed  as  important  risk  reducers  in  the 
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test  program  and  initial  flights  provided  valuable  and  useful 
information,  according  to  program  and  contractor  officials. 

•  All  test  aircraft  were  in  manufacturing  during  2007.  Low-rate  initial 
production  of  the  first  two  production  aircraft  and  advance  buys  for 
the  second  production  lot  also  got  under  way. 


Development 
Program  Faces 
Increased  Risks  of 
Further  Cost 
Increases  and  More 
Schedule  Delays 


Late  in  2007,  DOD  officials  approved  a  risky  and  controversial  plan  that 
replenishes  management  reserves  by  reducing  development  test  aircraft 
and  test  flights  in  order  to  stay  within  current  cost  and  schedule  estimates 
Difficulties  in  stabilizing  aircraft  designs  and  inefficient  production  of  test 
aircraft  resulted  in  spending  management  reserves  faster  than  anticipated. 
The  flight  test  program  has  barely  begun,  but  faces  substantial  risks  with 
reduced  assets  as  delays  in  design  and  manufacturing  continue  to  further 
compress  the  time  available  to  complete  development  work  prior  to 
operational  testing  and  to  support  the  full-rate  production  decision.  The 
JSF  program  is  halfway  to  its  planned  completion,  but  is  behind  schedule 
and  over  cost.  On  the  basis  of  evidence  we  have  gathered,  development 
costs  can  be  expected  to  increase  substantially  from  the  current  reported 
program  estimate,  and  the  time  needed  to  complete  development  testing 
and  subsequent  initial  operational  testing  will  likely  need  to  be  extended, 
delaying  the  full-rate  production  decision  now  planned  for  October  2013. 


Plan  to  Address 
Management  Reserve 
Depletion  Adds  Risk  to  the 
JSF  Development  Effort 


The  Office  of  the  Secretary  of  Defense  (OSD)  approved  the  Mid-Course 
Risk  Reduction  plan  in  September  2007.  The  plan  reduces  development 
test  aircraft  and  test  flights,  and  accelerates  the  reduction  of  the 
contractor’s  development  workforce  in  order  to  restore  management 
reserves  to  the  level  considered  prudent  to  complete  the  development 
contract  as  planned  and  within  the  current  cost  estimate.  The  test 
community  and  others  within  DOD  believe  the  plan  puts  the  development 
flight  program  at  considerable  risk  and  trades  known  cost  risk  today  for 
unknown  cost  and  schedule  risk  in  the  future. 


Management  reserves  are  budgeted  funds  set  aside  for  unanticipated 
development  challenges  and  increase  a  program’s  capacity  to  deal  with 
unknowns.  At  development  start,  JSF  budgeted  reserves  at  10  percent  of 
contract  value  and  expected  to  draw  on  them  at  about  the  same  rate  as 
contract  execution.  However,  the  program  has  had  to  use  these  funds 
much  faster  than  expected  to  pay  for  persistent  development  cost 
increases  and  schedule  delays.  A  combination  of  factors  contributed  to 
this  problem,  such  as  late  release  of  engineering  drawings,  production 
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taking  longer  than  planned,  and  late  delivery  of  parts  from  suppliers.  In 
turn,  these  contributed  to  continuing  cost  and  schedule  impacts  in  the 
manufacture  of  development  test  aircraft,  including  extensive  and 
inefficient  out-of-station  work  and  delays  in  proving  out  the  production 
schedule.  Figure  2  shows  how  management  reserves  totaling  almost  $1.4 
billion  have  been  depleted  since  the  2004  Replan. 


Figure  2:  JSF’s  Use  of  Management  Reserves 


$430  million  for  engineering  drawings 
$370  million  for  supplier  design  and  performance 
$350  million  for  production  materials  and  labor 
$163  million  for  other 

$160  million  for  weight  and  technical  changes 


Source:  GAO  analysis  of  DOD  data. 


By  mid-2007,  the  development  program  had  completed  one-half  of  the 
amount  of  work  scheduled,  but  had  expended  two-thirds  of  the  budget. 
Management  reserves  had  shrunk  to  about  $400  million,  less  than  one-half 
the  amount  officials  believed  necessary  to  complete  the  final  6  years  of 
development.  At  the  same  time,  the  program  faced  significant 
manufacturing  and  software  integration  challenges,  costly  flight  testing, 
and  $950  million  in  other  known  cost  risks.  This  presented  the  program 
with  a  likely  untenable  contract  overrun  sometime  in  2008  if  no  action  was 
taken.  JSF  program  management  identified  a  continuing  persistent  cost 
variance  of  $250  million  to  $300  million  in  the  aircraft  development 
contract  and  the  associated  shortfall  in  reserves  that  required  near-term 
action  beyond  “belt  tightening.” 
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An  overarching  integrated  product  team  considered  several  alternative 
actions,  including  doing  nothing  and  adding  funds  from  procurement,  but 
the  team  chair  concluded  that  replenishment  of  the  management  reserve 
was  essential  to  position  the  JSF  program  to  successfully  address  its 
anticipated  future  development  challenges.  This  option,  dubbed  the  Mid- 
Course  Risk  Reduction  Plan,  removed  two  development  aircraft  (one 
CTOL  and  one  CV),  eliminated  approximately  850  test  flights  from  the 
current  test  plan,  revised  the  verification  strategy,  increased  the  use  of 
ground  test  labs  and  the  flying  test  bed,  and  maximized  the  number  of  test 
points  to  be  accomplished  during  test  flights.  The  plan  also  accelerated 
reductions  in  contractor  staff  and  took  other  actions.  In  total,  these 
planned  actions  are  expected  to  add  between  $470  million  and  $650 
million  into  the  reserve  to  recapitalize  it  to  about  $1  billion,  an  amount 
officials  believe  will  be  needed  to  complete  development.  Officials  intend 
to  use  reserves  to  recover  cost  and  schedule  losses  in  manufacturing  and 
to  cover  additional  future  needs. 

This  plan  was  subsequently  approved  by  OSD,  although  serious  risks  were 
acknowledged  and  the  team  was  divided  on  whether  the  added  risks 
outweighed  the  intended  benefit.  Those  in  favor  of  the  plan  believed  that 
actions  were  urgently  needed  to  fix  the  funding  imbalance  and  avoid  a 
contract  overrun.  In  this  view,  the  plan  would  serve  as  a  stopgap  measure 
to  delay  another  program  restructure  until  more  program  knowledge  and  a 
clearer  understanding  of  future  cost  requirements  were  gained. 

Officials  from  several  defense  offices  thought  the  risks  to  testing  were  too 
great  and  that  the  plan  did  not  address  the  underlying  design  and 
manufacturing  problems.  The  Director,  Operational  Test  and  Evaluation, 
identified  specific  risks  associated  with  the  revised  test  verification 
strategy  and  recommended  against  deleting  the  aircraft,  citing  inadequate 
capacity  to  handle  the  pace  of  mission  testing,  and  for  ship  suitability, 
signature  testing,  and  suitability  evaluations.  This  increased  the  likelihood 
of  not  finding  and  resolving  critical  design  deficiencies  until  operational 
testing,  when  it  is  more  costly  and  disruptive  to  do  so.  OSD’s  Systems  and 
Software  Engineering  office  concurred,  expressed  concerns  that  the  plan 
did  not  treat  the  root  causes  of  ongoing  production  problems,  and  doubted 
that  the  contractor  schedule  was  achievable.  The  Cost  Analysis 
Improvement  Group  and  others  agreed  that  there  was  too  much  risk  in 
reducing  test  assets  at  this  time  since  no  production  representative  variant 
had  started  flight  tests  and  no  analysis  of  the  management  reserve 
depletion  had  been  completed.  In  summary,  the  plan  trades  known  cost 
risk  today  for  unknown  cost  and  schedule  risk  in  the  future. 
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According  to  our  analysis  of  available  evidence,  manufacturing  test 
aircraft  continue  to  run  behind  schedule.  The  prime  contractor  has  revised 
the  test  aircraft  manufacturing  schedule  three  times,  resulting  in  slips  of 
up  to  16  months  in  first  flight  dates  of  test  aircraft.  To  date,  about  3 
months  of  progress  has  been  made  for  every  4  months  of  effort.  As 
officials  for  now  have  decided  not  to  extend  the  development  period  and 
delay  operational  tests  and  full-rate  production,  this  inefficiency  increases 
risk  and  further  compresses  time  and  assets  available  to  complete  test 
activities. 

Repercussions  from  the  late  release  of  engineering  drawings,  design 
changes,  and  parts  shortages  continue  to  cause  delays  and  force  inefficient 
production  line  workarounds  where  unfinished  work  is  completed  out  of 
station.6  Production  data  provided  by  the  Defense  Contract  Management 
Agency  (DCMA)  show  continuing  critical  part  shortages,  high  change 
traffic,  out-of-station  work,  quality  issues,  and  planning  rework.  These 
conditions  have  also  delayed  efforts  to  mature  and  demonstrate  the 
production  process  even  as  work  begins  on  the  first  production  lot.  The 
contractor  has  not  yet  proven  it  can  efficiently  build  the  JSF,  and  test 
aircraft  are  being  built  differently  from  the  process  expected  for  the 
production  aircraft. 

The  first  test  aircraft,  a  non-production-representative  conventional 
landing  prototype  completed  in  2006,  required  66,000  more  labor  hours 
(about  35  percent  more)  to  build  than  planned.  It  encountered  most  of  its 
inefficiencies  in  the  wing  and  final  assembly  phases.  The  second  test 
aircraft,  a  STOVL  model,  left  the  production  line  in  December  2007,  and  its 
first  flight  is  expected  in  May  2008,  8  months  later  than  originally 
scheduled.  It  cost  about  26-30  percent  more  to  build  than  planned. 
Contractor  data  show  that  the  wings  were  only  three-fifths  complete  when 
moved  to  final  assembly.  As  a  result,  over  26,000  more  labor  hours  had  to 
be  performed  out  of  station  to  complete  the  wing  assembly  for  this 
aircraft. 

Table  2  shows  work  performance  on  the  first  seven  test  aircraft  to  enter 
manufacturing.  (This  does  not  include  the  original  prototype  completed  in 


Manufacturing 
Inefficiencies  Continue  to 
Increase  Costs  and  Delay 
the  Production  of 
Development  Aircraft 


6  An  efficient  production  line  establishes  an  orderly  flow  of  work  as  a  product  moves  from 
workstation  to  workstation  and  on  to  final  assembly.  Out-of-station  work,  sometimes 
referred  to  as  traveled  work,  refers  to  completing  unfinished  work  on  major  components, 
e.g.,  the  wings,  after  they  have  left  the  wing  workstation  and  moved  down  the  production 
line  to  another  station. 


Page  14 


GAO-08-388  Joint  Strike  Fighter 


December  2006.)  These  data  show  that  nearly  all  aircraft  are  persistently 
behind  schedule  in  completing  work  on  these  three  critical  components  at 
the  Fort  Worth,  Texas,  facility.  In  terms  of  cost,  the  data  show  overall 
good  performance  in  constructing  the  forward  fuselage,  but  poor  results 
for  the  wing  and  final  assembly. 


Table  2:  Manufacturing  Performance  Data,  End  of  September  2007 


Development 

aircraft 

Forward  fuselage 

Wing 

Final  assembly 

Days  behind8 

Cost 

efficiency15 

Days  behind  Cost  efficiency 

Days  behind  Cost  efficiency 

STOVL-1 

In  mate 

119% 

In  mate 

69% 

77% 

STOVL-2 

-19 

148% 

-129 

65% 

69% 

STOVL-3 

-34 

133% 

-134 

49% 

- 

STOVL-4 

-68 

115% 

-162 

41% 

- 

CTOL  -1 

-73 

139% 

-279 

23% 

- 

CTOL-2 

-35 

78% 

-283 

- 

- 

CTOL-3 

-35 

58% 

-140 

- 

- 

Source:  GAO  analysis  of  Defense  Contract  Management  Agency  data 

“This  column  represents  the  number  of  days  the  aircraft  were  behind  in  manufacturing  and  assembly 
of  these  components. 

bThe  cost  efficiency  column  is  a  measure  of  how  well  the  component  is  actually  performing  financially 
in  relation  to  earned  budgets.  Cost  efficiency  of  100  percent  or  higher  indicates  good  performance; 
scores  under  100  indicate  production  inefficiencies. 

Because  of  production  inefficiencies  and  delays,  the  contractor  has  had  to 
lengthen  the  manufacturing  schedule  three  times  to  provide  more  time  to 
complete  work.  Production  line  problems  have  resulted  in  slips  of 
between  11  and  16  months  to  first  flight  dates  for  each  variant.  At  the  time 
of  our  review,  a  fourth  schedule  was  being  prepared  that  would  add 
another  1  to  4  months  to  schedules.  Officials  are  reporting  some 
improvements  in  parts  shortages,  assembly,  and  product  quality,  but 
expect  the  cascading  effects  from  the  design  delays  and  manufacturing 
inefficiencies  to  linger  for  another  couple  of  years. 


Development  Flight  Test 
Efforts  Are  Beginning  with 
Fewer  Assets  and  Revised 
Verification  Strategy 


The  flight  test  program  has  just  begun,  with  only  about  26  flights 
completed  as  of  January  2008.  The  program  had  originally  planned  to 
conduct  development  flight  tests  using  15  aircraft.  The  recent  decision  to 
reduce  test  aircraft  to  13  (including  the  prototype),  cut  back  the  number  of 
flights,  and  change  how  some  capabilities  are  tested  will  stress  resources, 
compress  time  to  complete  testing,  and  increase  the  number  of 
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development  test  efforts  that  will  overlap  the  planned  start  of  operational 
testing  in  October  2012.  Test  officials  are  concerned  that  capacity  will  be 
too  constrained  to  meet  schedules  and  to  adequately  test  and  demonstrate 
aircraft  in  time  to  support  operational  testing  and  the  full-rate  production 
decision  in  October  2013.  The  full  extent  of  changes  and  impacts  from  a 
revised  test  verification  strategy  are  still  evolving.  Program  officials 
reported  that  if  test  assets  become  too  constrained,  production  aircraft 
may  eventually  be  used  to  complete  development  testing. 

The  number  of  development  flight  tests  had  already  been  reduced  twice 
before  the  Mid-Course  Risk  Reduction  plan,  as  shown  in  figure  3.  Test 
flights  have  now  been  reduced  by  more  than  1,800  flights  (26  percent)  over 
the  last  2  years. 


Figure  3:  JSF  Development  Flight  Tests  Planned 

Number  of  flights 
8,000 


6,979 


Other  test  issues  and  events  included  the  following: 

•  Flight  tests  started  with  the  initial  development  test  aircraft,  which  is 
not  considered  to  be  a  production-representative  aircraft.  According  to 
program  officials,  initial  flights  of  this  aircraft  yielded  very  useful 
information  on  flight  characteristics.  However,  three  incidents — an 
electrical  flight  control  actuator  malfunction  in-flight  and  an  engine 
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blade  failure  during  a  ground  test — delayed  further  testing  from  May  to 
December  2007.  Another  blade  failure  occurred  in  February  2008. 

•  Initial  flights  of  the  Cooperative  Airborne  Test  Bed  aircraft  in  2007 
verified  its  airworthiness,  and  it  was  then  modified  to  integrate  some 
JSF  systems  hardware  and  software.  In  December  2007,  it  began  some 
limited  mission  flight  tests,  but  is  not  yet  fully  configured.  The  Mid- 
Course  Risk  Reduction  plan  revised  the  development  test  verification 
strategy  to  increase  reliance  on  this  specially  configured  aircraft  to  test 
capabilities  that  were  going  to  be  demonstrated  on  JSF  aircraft. 

•  An  operational  assessment  by  testers  from  the  Navy,  Air  Force,  and  the 
United  Kingdom’s  Royal  Air  Force  was  accomplished  from  March  2004 
to  December  2005  to  assess  development  progress  and  current  JSF 
mission  capability.  The  February  2006  report  concluded  that  the 
baseline  flight  test  schedule  provided  little  capability  to  deal  with 
unforeseen  problems  and  still  meet  the  scheduled  start  of  operational 
test  and  evaluation  in  October  2012.  Testing  officials  said  the  JSF  flight 
test  program  was  following  the  historical  pattern  of  legacy  programs  in 
making  overoptimistic  plans  and  using  assumptions  not  supported  by 
historical  data.  In  legacy  aircraft,  these  practices  resulted  in  capacity 
constraints,  program  slips,  and  reduced  testing  tasks.  We  note  that 
these  concerns  about  the  JSF  were  expressed  at  a  time  when  the  test 
program  was  expected  to  have  the  full  complement  of  15  test  aircraft, 
not  the  13  now  planned. 


Development  Challenges 
Have  Been  Exacerbated  by 
Inattention  to  Best 
Acquisition  Practices 


A  program  as  complex  and  technically  challenging  as  the  JSF  would  be 
expected  to  have  some  setbacks,  but  we  believe  that  the  cause  of  many 
cost  and  schedule  problems  can  be  traced  to  an  acquisition  strategy  and 
decisions  at  key  junctures  that  did  not  adequately  follow  the  best  practices 
we  have  documented  in  successful  commercial  and  government 
programs.7  The  JSF  started  system  development  before  requisite 
technologies  were  ready,  started  manufacturing  test  aircraft  before 
designs  were  stable,  and  moved  to  production  before  flight  tests  have 
adequately  demonstrated  that  the  aircraft  design  meets  performance  and 
operational  suitability  requirements.  We  previously  reported  that  the  JSF 
acquisition  strategy  incorporated  excessive  overlap  in  development  and 


'  For  an  overview  of  the  best  practices  methodologies  and  how  current  defense  programs 
fared,  see  our  report  of  last  year  on  major  acquisitions,  including  the  JSF,  in  GAO,  Defense 
Acquisitions:  Assessments  of  Selected  Weapon  Proqrams,  GAO-07-406SP  (Washington, 
D.C.:  Mar.  30,  2007). 
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production,  posing  substantial  risks  for  cost  overruns,  schedule  slips,  and 
late  delivery  of  promised  capabilities  to  the  warfighter.8 

Six  years  after  system  development  start,  only  two  of  the  JSF’s  eight 
critical  technologies  are  mature  by  best  practice  standards,  three  are 
approaching  maturity,  and  three  are  immature.  Maturing  critical 
technologies  during  system  development  led  to  cost  growth.  For  example, 
development  costs  for  the  electric-hydraulic  actuation  and  power  thermal 
management  systems  have  increased  by  195  and  93  percent  respectively 
since  2003. 

All  three  variants  fell  significantly  short  of  meeting  the  best  practices 
standard  of  90  percent  of  drawings  released  at  the  times  of  their  respective 
critical  design  reviews:  46  percent  for  the  STOVL,  43  percent  for  the  CV, 
and  3  percent  for  the  CTOL.  Design  delays  and  changes  to  designs  were 
cited  by  the  Mid-Course  Risk  Reduction  team  as  the  precipitating  cause 
leading  to  the  depletion  of  management  reserves.  The  late  release  of 
drawings  resulted  in  a  cascading  of  problems  in  establishing  suppliers  and 
manufacturing  process,  which  led  to  late  parts  deliveries,  delayed  the 
program  schedule,  and  forced  inefficient  manufacturing  processes  to 
compensate  for  the  delay. 

Also,  the  program  began  initial  low-rate  production  in  2007  before 
delivering  an  aircraft  that  fully  represents  the  expected  design.  Efforts  to 
mature  production  are  constrained  because  the  designs  are  not  fully 
proven  and  tested,  and  manufacturing  processes  are  not  demonstrated.  A 
fully  integrated,  capable  production  aircraft  is  not  expected  to  enter  flight 
testing  until  fiscal  year  2012,  increasing  risks  that  problems  found  may 
require  redesign,  production  line  changes,  and  retrofit  expenses  for 
aircraft  already  built. 


Program  Costs  Expected 
to  Increase  and  Schedule 
Worsen 


On  the  basis  of  the  evidence,  we  expect  JSF  program  costs  to  increase  and 
the  schedule  worsen  to  the  point  where  the  development  period  will  likely 
need  to  be  extended  and  Initial  Operational  Test  and  Evaluation  (IOT&E) 
and  full-rate  production  delayed.  A  major  program  restructure  seems 
inevitable,  unless  significant  elements  of  the  program  can  be  safely 
eliminated  or  deferred.  The  Mid-Course  Risk  Reduction  plan  does  not 
directly  address  design  and  manufacturing  inefficiencies  that  created  the 


8  GAO-05-271. 
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problem  in  the  first  place.  If  the  root  causes  are  not  identified  and  fixed, 
the  rapid  depletion  of  management  reserves  can  be  expected  to  continue, 
and  more  funding  will  be  needed  to  complete  development. 

There  is  no  reason  to  believe  that  these  problems  can  be  easily  and  quickly 
fixed.  While  there  have  been  some  assembly  line  improvements,  program 
officials  expect  the  manufacturing  problems  to  persist  for  about  2  more  years. 
Officials  hope  this  plan  will  give  them  a  period  of  time  to  better  and  more  fully 
assess  all  the  issues  and  reevaluate  development  costs  and  schedule 
requirements.  They  are  depending  on  the  revised  test  verification  plans  to 
maintain  the  pace  and  efficacy  of  development  testing,  but  the  test  community 
is  dubious.  What  seem  more  likely  are  additional  costs  and  time  to  overcome 
inadequate  capacity  and  the  elimination  or  deferral  of  more  test  activities. 
Eliminating  development  test  activities  and  deferring  additional  tasks  to  be 
completed  during  operational  testing  increase  the  likelihood  that  design  and 
performance  problems  will  not  be  identified  and  resolved  until  late  in  the 
program,  when  it  is  more  costly  and  disruptive  and  could  delay  the  delivery  of 
capabilities  to  the  warfighter. 

There  are  also  abundant  other  indicators  that  acquisition  costs  will 
substantially  increase  from  what  is  now  being  reported  to  Congress. 
Specifically: 

•  DOD  has  identified  billions  of  dollars  in  unfunded  requirements  that 
are  not  in  the  program  office  estimate,  including  additional  tooling  and 
procurement  price  hikes. 

•  A  new  manufacturing  schedule  in  the  works  indicates  continued 
degradation  in  the  schedule  and  further  extends  times  for  first  flights. 

•  Both  the  aircraft  and  engine  development  contracts  have  persistent, 
substantial  cost  variances  that  cost  analysts  believe  are  too  large  and 
too  late  in  the  program  to  resolve  without  adding  to  budget. 

•  The  prime  contractor  and  program  office  are  readying  a  new  estimate 
at  completion,  which  is  expected  to  be  much  larger  than  what  is  now 
budgeted. 

•  Three  defense  organizations  independent  of  the  JSF  program  office 
have  all  concluded  that  the  program  office’s  cost  estimate  is 
significantly  understated  and  the  current  schedule  unlikely  to  be 
achieved. 
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For  these  and  other  reasons,  we  believe  that  the  current  JSF  cost  and 
schedule  reported  to  Congress  are  not  reliable  for  decision  making,  as 
discussed  next. 


JSF  Program  Cost 
Estimate  Is  Not 
Reliable 


The  $299.8  billion  acquisition  cost  estimate  for  the  JSF  program  is  not 
reliable  because  it  is  not  sufficiently  comprehensive,  accurate, 
documented,  or  credible.  GAO’s  Cost  Assessment  Guide  outlines  best 
practices  used  throughout  the  federal  government  and  industry  for 
producing  reliable  and  valid  cost  estimates.  We  assessed  the  cost¬ 
estimating  methodologies  used  by  the  JSF  program  office  against  these 
best  practices  and  determined  that  certain  key  costs  were  excluded, 
assumptions  used  were  overly  optimistic,  documentation  was  inadequate, 
and  no  analysis  had  been  done  to  state  the  confidence  and  certainty  the 
program  office  had  in  its  cost  estimate.  As  a  result  of  these  weaknesses, 
the  JSF  program  acquisition  cost  estimate  is  not  reliable  for  decision 
making.  Appendix  II  contains  a  more  detailed  discussion  of  the  specific 
shortcomings  we  and  the  other  DOD  organizations  have  found  in  the 
program  office  cost-estimating  methodologies  and  their  potential  impacts. 


The  JSF  Cost  Estimate  Is  Estimates  are  comprehensive  when  they  contain  a  level  of  detail  that 
Not  Comprehensive  ensures  that  all  pertinent  costs  are  included  and  no  costs  are  double- 

counted.  It  is  important  to  ensure  the  completeness,  consistency,  and 
realism  of  the  information  contained  in  the  cost  estimate.  Our  review  of 
the  JSF  development  cost  estimate  showed  that  there  are  several  cost 
categories  totaling  more  than  $10  billion  that  are  excluded  or 
underreported  in  the  program  office  estimate.  These  items  are  summarized 
in  table  3  below. 


Table  3:  Costs  Excluded  from  the  Program  Office  Acquisition  Cost  Estimate 

Cost  item 

Possible  impact 

Alternate  engine 

$6.8  billion 

Military  construction 

$1.5  billion 

Tooling 

$2.1  billion 

Capabilities  dropped  from  development 

Unknown 

Total 

$10  billion  plus 

Source:  GAO  analysis  of  DOD  data 
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Specifically: 

•  The  current  acquisition  cost  estimate  includes  only  near-term 
development  funding  for  the  alternate  engine  program,  excluding 
procurement-related  and  other  development  costs  of  about  $6.8  billion. 

•  The  military  services  have  not  firmly  established  basing  needs  for  the 
entire  planned  JSF  force,  but  an  earlier  top-line  estimate  for  military 
construction  was  at  least  $2  billion.  The  current  total  cost  estimate 
includes  only  near-term  budgeted  costs  of  $533  million. 

•  The  JSF  program  recently  increased  its  estimate  of  tooling  costs  by 
$2.1  billion  due  to  the  inclusion  of  additional  tooling  requirements  and 
estimating  methodology  changes. 

•  Cost  and  performance  trade-offs  during  development  deferred  some 
requirements  from  the  current  program  that  may  later  require 
additional  funding.  The  program  office  has  not  quantified  these 
deferrals,  but  Naval  Air  Systems  Command  (NAVAIR)  officials  told  us 
that  the  amount  could  be  in  the  billions  of  dollars. 


The  JSF  Cost  Estimate  is  Estimates  are  accurate  when  they  are  based  on  an  assessment  of  the  costs 

Not  Accurate  most  likely  to  be  incurred.  Therefore,  when  costs  change,  best  practices 

require  that  the  estimate  be  updated  to  reflect  changes  in  technical  or 
program  assumptions  and  new  phases  or  milestones.  DOD’s  Cost  Analysis 
Improvement  Group  (CAIG)  found  that  the  assumptions  the  JSF  program 
office  used  for  weight  growth,  staffing  head  counts,  commonality  savings 
for  similar  parts,  and  outsourced  labor  rate  savings  were  overly  optimistic 
and  not  supported  by  historical  data. 9  For  example,  the  program  office  had 
used  a  3  percent  factor  for  weight  growth  whereas  the  CAIG  used  a  6 
percent  factor  more  in  line  with  historical  data  from  other  programs.  With 
three  variants,  a  joint  program  with  international  participation,  three 
different  engines  (cruise,  second  engine,  and  lift)  in  development,  and 
more  than  double  the  amount  of  operational  flight  software  lines  of  code 
than  the  F-22A  and  more  than  four  times  that  of  the  F/A-18E/F,  the  JSF 


9  The  CAIG  serves  as  the  principal  advisory  body  to  the  milestone  decision  authority  on  all 
matters  concerning  an  acquisition  program’s  life-cycle  cost,  and  is  given  general 
responsibilities  for  establishing  DOD  policy  guidance  on  a  number  of  matters  relating  to 
cost  estimating.  The  independent  CAIG  cost  estimate  is  designed  to  assess  the  program 
office  estimate  and  ensure  realistic  cost  estimates  are  considered. 
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program  is  substantially  more  complex  than  the  F-22  or  F/A-18E/F,  and 
therefore  may  not  merit  assumptions  that  are  even  as  optimistic  as  the 
historical  data  for  those  programs. 

The  program  cost  estimate  is  also  considered  inaccurate  because  it  relies 
on  data  and  reports  found  to  be  deficient.  JSF  program  office  used 
Lockheed  Martin  earned  value  management  (EVM)  data  in  estimating 
development  costs.10  However,  DCMA  determined  that  the  data  as  being  of 
very  poor  quality  and  issued  a  report  in  November  2007  stating  that  it  is 
deficient  to  the  point  where  the  government  is  not  obtaining  useful 
program  performance  data  to  manage  risks.  Among  other  problem  areas, 
DCMA  found  that  the  contractor  was  using  management  reserve  funds  to 
alter  its  own  and  subcontractor  performance  levels  and  cost  overruns. 
DCMA  officials  who  conducted  the  review  told  us  that  the  poor  quality  of 
the  data  invalidated  key  performance  metrics  regarding  cost  and  schedule, 
as  well  as  the  contractor’s  estimate  of  the  cost  to  complete  the  contract.  At 
the  time  of  our  review,  corrective  actions  and  plans  were  in  process. 


The  JSF  Cost  Estimate  Is  Cost  estimates  are  well  documented  when  they  can  be  easily  repeated  or 

Not  Well  Documented  updated  and  can  be  traced  to  original  sources  through  auditing.  Rigorous 

documentation  increases  the  credibility  of  an  estimate  and  helps  support 
an  organization’s  decision-making  process.  The  documentation  should 
explicitly  identify  the  primary  methods,  calculations,  results,  rationales, 
assumptions,  and  sources  of  the  data  used  to  generate  each  cost  element. 
All  the  steps  involved  in  developing  the  estimate  should  be  documented  so 
that  a  cost  analyst  unfamiliar  with  the  program  can  recreate  the  estimate 
with  the  same  result. 

We  found  that  the  JSF  cost  model  is  highly  complex  and  the  level  of 
documentation  is  not  sufficient  for  someone  unfamiliar  with  the  program 
to  easily  recreate  it.  Specifically,  we  found  that  the  program  office  does 
not  have  formal  documentation  for  the  development,  production,  and 
operation  and  support  cost  models  and  could  not  provide  detailed 
documentation  such  as  quantitative  analysis  to  support  its  assumptions. 
For  the  development  cost  estimate,  the  JSF  program  officials  said  they  did 
not  have  a  cost  model  that  was  continually  updated  with  actual  costs. 


10  Earned  value  management  is  a  method  of  tracking  and  measuring  the  value  of  work 
accomplished  in  a  given  period  and  comparing  it  with  the  planned  value  of  work  scheduled 
and  the  actual  cost  of  work  accomplished.  Its  use  is  required  by  federal  regulations. 
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Instead  the  program  office  relies  heavily  on  earned  value  management 
data  and  contractor  analysis  to  update  its  development  cost  estimate. 


The  JSF  Cost  Estimate  Is  Estimates  are  credible  when  they  have  been  cross-checked  with  an 
Not  Credible  independent  cost  estimate  and  when  a  level  of  uncertainty  associated  with 

the  estimate  has  been  identified.  An  independent  cost  estimate  provides 
the  estimator  with  an  unbiased  test  of  the  reasonableness  of  the  estimate 
and  reduces  the  cost  risk  associated  with  the  project  by  demonstrating 
that  alternative  methods  generate  similar  results. 

Several  independent  organizations  have  reviewed  the  JSF  program  and  are 
predicting  much  higher  costs  than  the  program  office.  Table  4  below 
provides  a  summary  of  these  assessments. 


Table  4:  Outside  Organizations’  Assessments  of  JSF  Cost  and  Schedule 

Assessing 

organization 

Impact  on  cost 

Impact  on  schedule 

CAIG 

$5.1  billion  more  for  development,  over 
$33  billion  more  for  procurement 

12  months  slip 

NAVAIR 

$8  billion  to  $13  billion  additional 
development  costs  or  trade-offs  adding 
to  procurement  costs 

19-27  months  slip 

DCMA 

$4.9  billion  additional  cost  to  complete 
Lockheed  Martin  development  contract 

Up  to  12  months  slip 

Source:  CAIG,  NAVAIR,  DCMA. 


CAIG  estimates  were  prepared  using  different  and  more  realistic 
assumptions  and  schedule  projections  than  the  program  office  estimate. 
NAVAIR,  which  provides  resources  to  the  JSF  program  office  cost¬ 
estimating  function,  derived  much  higher  cost  estimates  and  a  longer 
development  period  based  on  historical  cost  performance  and  removing 
what  it  considered  to  be  artificial  and  unachievable  schedule  constraints. 
Officials  were  also  concerned  about  the  amount  and  future  impact  of 
requirements  potentially  traded  or  pushed  off  into  the  procurement  phase, 
which  could  be  even  more  costly.  DCMA  projected  higher  development 
costs  for  the  aircraft  contract  based  on  adjusted  cost  and  schedule 
performance  to  date  and  assuming  additional  slips.  Officials  continue  to 
examine  the  contractor’s  deficient  earned  value  management  system  and 
its  misreporting  of  cost  and  schedule  data. 
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DOD  Intends  to  Wait  to  The  JSF  program  has  not  conducted  a  fully  documented  independent  cost 

Make  a  New  and  estimate  since  system  development  start  in  2001.  Despite  reliability 

Independent  Cost  Estimate  concerns  and  all  the  significant  events  and  changes  in  cost,  schedule,  and 

quantity  since  then — those  reported  by  the  program  office  as  well  as  those 
identified  by  other  defense  organizations  and  us — DOD  does  not  intend  to 
accomplish  another  one  until  required  to  support  the  full-rate  production 
decision  in  2013.  If  so,  this  will  mean  that  the  program — DOD’s  largest 
acquisition  and  vitally  important  to  our  allies — will  have  a  12-year  gap 
between  official  validated  cost  estimates.  The  program  may  complete 
development  and  be  6  years  into  production  before  an  accurate,  up-to- 
date,  and  reliable  official  cost  estimate  is  done. 

Despite  widely  held  views  that  costs  will  likely  be  higher  and  the  schedule 
longer  than  reported,  the  JSF  program  continues  to  be  funded  to  the  level 
of  the  program  office  estimate.  DOD  acquisition  policy  requires  fully 
documented  total  program  life-cycle  cost  estimates,  with  validation  by  the 
CAIG,  at  certain  major  decision  points  and  when  mandated  by  the 
milestone  decision  authority.  DOD  officials  decided  not  to  do  such  an 
estimate  at  the  start  of  low-rate  initial  production  in  2007,  which  typically 
coincides  with  a  major  milestone. 


Future  Challenges  as 
Program  Moves 
Forward 


The  JSF  is  entering  its  most  challenging  phase  as  it  finalizes  three  designs, 
matures  manufacturing  processes,  conducts  flight  tests,  and  ramps  up 
production.  The  first  and  foremost  challenge  is  maintaining  affordability  in 
three  dimensions — reasonable  procurement  prices,  stable  annual  funding, 
and  economical  life-cycle  operating  and  support  costs.  If  affordability  is 
not  maintained  during  the  acquisition  program,  quantities  bought  by  the 
United  States  and  allies  may  either  decrease  or  else  consume  more  of  the 
available  defense  budgets.  Over  the  life  cycle  of  a  system,  higher  costs  for 
maintaining  readiness  and  maintainability  drive  up  annual  operating 
expenses  and  may  limit  funds  for  new  investments.  Other  program 
challenges  could  affect  future  quantities  and  the  mix  of  aircraft  procured 
by  the  United  States  and  our  allies. 


Affordability  Concerns  From  its  outset,  the  JSF  goal  has  been  to  develop  and  field  an  affordable, 

Have  Major  Repercussions  highly  common  family  of  strike  aircraft.  Rising  unit  procurement  prices, 

and  somewhat  lower  commonality  than  expected,  raise  concerns  that  the 
United  States  and  its  allies  may  not  be  able  to  buy  as  many  aircraft  as 
currently  planned.  Average  unit  procurement  costs  are  up  27  percent  since 
the  2004  Replan  and  51  percent  since  the  start  of  system  development  (see 
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table  1).  Rising  prices  erode  buying  power,  likely  resulting  in  reduced 
quantities  and  delays  in  delivering  promised  capabilities  to  the  warfighter. 

The  program  also  places  an  unprecedented  demand  for  funding  on  the 
defense  budget — an  annual  average  of  about  $11  billion  for  the  next  two 
decades-with  attendant  funding  risk  should  political,  economic  or  military 
conditions  change.  The  JSF  will  have  to  annually  compete  with  other 
defense  and  nondefense  priorities  for  the  shrinking  discretionary  federal 
dollar.  To  complete  the  acquisition  program  as  currently  planned,  JSF  will 
require  about  $269  billion  from  2008  through  2034.  Annual  funding 
requirements  for  procurement  increase  rapidly  as  production  ramps  up  to 
the  full-rate  production  decision  expected  in  October  2013.  During  the 
peak  years  of  production,  JSF  procurement  funding  requirements  are 
expected  to  average  about  $12.6  billion  per  year  for  the  12-year  period 
spanning  fiscal  years  2012-2023. 

Figure  4  illustrates  the  annual  funding  requirements  as  of  December  2006 
and  contrasts  these  with  plans  from  prior  years.  The  December  2003  line 
shows  the  funding  profile  resulting  from  the  2004  Replan  and  the  2006  line 
shows  the  jump  in  funding  needed  to  accommodate  program  cost 
increases  in  the  period  following  the  Replan.  The  2006  data  reflect  the 
impact  on  annual  funding  requirements  from  extending  procurement  7 
years.  The  extension  reduced  annual  budget  amounts,  but  requires 
continued  funding  through  2034  to  procure  deferred  quantities.  DOD 
calculated  that  the  extension  added  $11.2  billion  to  total  procurement 
cost. 
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Figure  4:  JSF  Acquisition  Program’s  Annual  Funding  Requirements 

Funding  requirements  (dollars  in  billions) 

18 

16 


^  —  December  2003  plan 
December  2005  plan 
December  2006  plan 

Source:  GAO  analysis  of  DOD  data. 


A  third  aspect  of  affordability  is  the  life-cycle  cost  of  ownership.  DOD  is 
recapitalizing  its  tactical  air  forces  by  replacing  aging  legacy  systems  with 
new,  more  capable  systems,  like  the  JSF,  that  incorporates  reliability  and 
maintainability  features  designed  to  reduce  future  operating  costs. 
Recently,  DOD  sharply  increased  its  projection  of  JSF  operating  and 
support  costs  compared  to  previous  estimates.  The  December  2006  SAR 
projected  life-cycle  operating  and  support  costs  for  all  three  variants  at 
$660.3  billion,  almost  double  the  $346.7  billion  amount  shown  in  the 
December  2006  SAR  and  similar  earlier  estimates.  The  operating  cost  per 
flying  hour  for  the  JSF  CTOL  is  now  estimated  to  be  greater  than  current 
flying  hour  cost  for  the  F-16,  one  of  the  legacy  aircraft  to  be  replaced. 

Officials  explained  that  the  amounts  reported  in  2006  and  before  were 
early  estimates  based  on  very  little  data,  whereas  the  new  estimate  is  of 
higher  fidelity,  informed  by  more  information  as  JSF  development 
progresses  and  more  knowledge  is  obtained.  Factors  responsible  for  the 
increased  cost  estimate  included  a  revised  fielding  and  basing  plan, 
changes  in  repair  plans,  revised  costs  for  depot  maintenance,  increased 
fuel  costs,  increased  fuel  consumption,  revised  estimates  for  manpower 
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and  mission  personnel,  and  a  new  estimate  of  the  cost  of  the  JSF’s 
autonomic  logistics  system. 

Overall,  the  cost  of  ownership  represents  a  very  large  and  continuing 
requirement  for  the  life  of  fielded  aircraft.  According  to  the  new  estimate, 
we  calculate  that  DOD  will  incur  about  $24  billion  per  year  to  operate  and 
support  JSF  units,  assuming  the  quantities  now  planned  and  an  8,000-hour 
service  life  for  each  JSF  aircraft  fielded  over  time. 


Commonality  Is  Less  than 
Expected 


From  the  inception  of  the  program,  DOD  has  anticipated  major  cost 
savings  from  developing  and  fielding  JSF  variants  that  share  many 
common  components  and  subsystems.  While  a  degree  of  commonality  has 
been  achieved,  expectations  are  now  lower  than  they  were  at  program 
start.  Substantial  commonality  has  been  maintained  for  the  mission 
systems  among  all  three  variants  and  for  the  propulsion  system  of  the 
conventional  and  carrier  variants.  However,  commonality  among 
airframes  and  vehicle  systems  has  declined  overall  since  the  start  of 
system  development.  Figure  5  shows  the  decline  in  airframe  commonality, 
the  most  costly  of  the  four  major  categories.  For  example,  in  October  2001 
DOD  anticipated  that  the  CTOL  airframe  would  be  more  than  60  percent 
common  with  the  other  variants.  Commonality  had  declined  to  about  40 
percent  by  December  2006.  Lesser  commonality  will  likely  increase 
acquisition  and  future  support  costs. 
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Figure  5:  Changes  in  Airframe  Commonality 
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Source:  GAO  analysis,  DOD  data. 


Navy  and  Marine  Corps  The  current  JSF  program  shows  a  total  quantity  of  680  aircraft  to  be 
Requirements  and  Mix  Are  procured  by  the  Department  of  the  Navy,  but  the  allocation  between  the 
Unsettled  CV  and  STOVL  variants  has  not  been  officially  established.  We  observe 

that  the  Navy  and  Marine  Corps  have  somewhat  divergent  views  on  the 
quantities,  intended  employment,  and  basing  of  JSF  aircraft.  The  Navy 
wants  the  Marine  Corps  to  buy  some  CV  variants  and  continue  to  man 
some  of  its  carrier-based  squadrons.  The  Marine  Corps,  however,  wants  to 
have  a  future  strike  force  composed  solely  of  the  STOVL  variant  and  has 
established  a  requirement  for  420  aircraft.  During  conflicts,  the  Marines 
plan  to  forward  deploy  JSFs  to  accompany  and  support  the  expeditionary 
ground  forces. 

Navy  officials  told  us  that  they  have  some  time  to  make  decisions  because 
they  will  be  buying  a  mix  of  both  CVs  and  STOVLs  in  the  early  years  of 
production  and  that  funding  requirements  are  not  significantly  affected 
since  unit  prices  for  both  variants  are  about  the  same.  However,  we 
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believe  the  continuing  disagreements  on  basing,  employment,  and  force 
mix  will  have  increasingly  stronger  impacts  on  JSF  plans,  costs,  and 
international  partner  relations.  Decreased  quantities  of  STOVLs  bought  by 
the  Department  of  the  Navy  would  likely  result  in  higher  unit  prices  paid 
by  the  Marine  Corps  and  two  allies  buying  STOVLs.  Fundamental 
decisions  on  the  mix  of  naval  aircraft  also  affect  future  operating  and 
support  costs,  military  construction,  and  carrier  requirements. 

Officials  also  have  some  reservations  whether  they  can  afford  the 
quantities  now  planned  at  peak  production  rates.  Navy  and  Marine  Corps 
officials  told  us  last  year  that  buying  the  JSF  at  the  current  planned  rate — 
requiring  a  ramp-up  to  50  CV  and  STOVL  aircraft  by  fiscal  year  2015 — will 
be  difficult  to  achieve  and  to  afford,  particularly  if  costs  increase  and 
schedules  slip.  Officials  told  us  that  a  maximum  of  35  per  year  was 
probably  affordable,  given  budget  plans  at  that  time.11 


Containing  Future  Weight 
Growth 


While  the  weight  increase  has  been  addressed  for  now,  projections  are 
that  the  aircraft  weight  will  continue  to  increase  during  the  balance  of  the 
development  period,  consistent  with  weight  increases  seen  on  legacy 
aircraft  programs.  According  to  an  OSD  official  with  knowledge  of  legacy 
aircraft  development  efforts,  half  of  all  weight  growth  during  the 
development  effort  can  be  typically  expected  after  first  flight  but  prior  to 
initial  operational  capability,  and  that  additional  small  but  persistent 
weight  increases  can  be  expected  during  the  aircraft’s  service  life.  First 
flight  of  a  production-representative  JSF  has  not  yet  occurred,  and  weight 
is  running  very  close  to  the  limits  as  evaluated  by  engineering  analyses  and 
trend  extrapolation.  As  designs  continue  to  mature  and  flight  testing 
intensifies,  maintaining  weight  within  limits  to  meet  warfighter  capability 
requirements  will  be  a  continuing  challenge  and  pose  a  major  risk  to 
meeting  cost,  schedule,  and  performance  goals. 


Weight  growth  was  the  most  significant  challenge  faced  by  the  JSF 
program  early  in  development.  Redesign  efforts  to  address  weight  growth 
was  the  single  largest  factor  causing  the  $10  billion  cost  increase  and  18- 
month  extension  in  the  development  schedule  since  the  start  of  system 
development. 


11  GAO,  Tactical  Aircraft:  DOD  Needs  a  Joint,  and  Integrated  Investment  Strategy, 
GAO-07-415  (Washington,  D.C.:  Apr.  2,  2007). 
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Conclusions 


The  clear  implication  from  performance  to  date  and  the  Mid-Course  Risk 
Reduction  plan  is  that  additional  costs  and  time  will  be  needed  to 
complete  JSF  development.  The  plan  to  recapitalize  management  reserve 
at  the  expense  of  test  assets  is  risky  with  potential  major  impacts  down 
the  road  on  costs,  performance  requirements,  and  fielding  schedules.  The 
remaining  development  effort  will  be  less  robust  than  originally  planned 
and  depends  on  a  revised  test  verification  strategy  that  is  still  evolving.  As 
a  result,  the  development  effort  has  an  increased  risk  of  not  fully 
measuring  JSF  capabilities  and  deficiencies  prior  to  operational  testing 
and  could  result,  in  the  words  of  one  DOD  official,  in  the  future 
operational  test  period  being  one  of  discovery  rather  than  validation  of  the 
aircraft’s  capabilities  and  deficiencies.  Finding  and  fixing  deficiencies 
during  operational  testing  and  after  production  has  ramped  up  is  costly, 
disruptive,  and  delays  getting  new  capabilities  to  the  warfighter. 

Because  the  program  cost  estimate  is  not  reliable  when  judged  against 
best  standards,  the  decision  making  and  oversight  by  Congress,  top 
military  leaders,  and  our  allies  are  diminished.  The  picture  they  do  have  is 
one  where  costs  continue  to  rise  and  schedules  slip.  The  situation  will  be 
considerably  worsened  if  the  cost  estimates  of  defense  offices  outside  the 
program  are  more  accurate  than  the  conservative,  official  in-house 
estimates.  Waiting  12  years  between  fully  documented  and  validated  total 
program  cost  estimates  is  contrary  to  policy  and  good  management,  given 
all  the  changes  in  cost,  quantity,  schedules,  and  other  events  that  have 
occurred  since  the  2001  estimate.  The  size  of  the  JSF  acquisition,  its 
impact  on  our  and  allied  tactical  air  forces,  and  the  unreliability  of  the 
current  estimate  argue  for  an  immediate  new  and  independent  cost 
estimate  and  uncertainty  analysis.  This  is  critical  information  needed  by 
DOD  management  to  make  sound  trade-off  decisions  against  competing 
demands  and  by  Congress  to  perform  oversight  and  hold  DOD 
accountable. 

Program  problems  and  setbacks  must  be  put  into  perspective:  The  JSF  is 
DOD’s  largest  and  most  complex  aircraft  acquisition  and  an  integral 
component  of  the  future  force.  Problems  happen  in  such  an  environment. 
Progress  has  been  made  and  some  significant  challenges  overcome,  but 
more  await  as  program  moves  into  flight  testing  and  low-rate  production. 
Maintaining  affordability  so  the  United  States  military  and  our  allies  can 
buy,  field,  and  support  the  numbers  needed  by  the  warfighter  remains  the 
overarching  challenge. 
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Recommendations  for 
Executive  Action 


Because  of  the  elevated  risks  and  the  valid  objections  raised  by  the  test 
community  and  other  DOD  offices,  we  recommend  that  the  Secretary  of 
Defense  direct  elements  of  the  department  to  revisit  and,  if  appropriate, 
revise  the  Mid-Course  Risk  Reduction  plan  recently  approved.  This  should 
be  supported  by  an  intensive  analysis  that  includes  causes  of  management 
reserve  depletion,  an  evaluation  of  progress  against  the  baseline 
manufacturing  schedule,  and  the  progress  made  in  correcting  deficiencies 
in  the  contractor’s  earned  value  management  system.  It  should  also 
include  an  in-depth  examination  of  alternatives  to  the  current  plan  and 
address  the  specific  concerns  raised  by  officials  regarding  testing  capacity, 
the  integration  of  ground  and  flight  tests,  and  backup  plans  should 
capacity  become  overloaded. 

So  that  DOD  may  have  an  accurate  picture  of  JSF  cost  and  schedule 
requirements,  and  that  Congress  may  have  an  accurate  understanding  of 
future  funding  requirements,  we  recommend  that  the  Secretary  of  Defense 
direct  that 

1.  The  JSF  program  office  update  its  cost  estimate  using  best  practices, 
so  that  the  estimate  is  comprehensive,  accurate,  well  documented,  and 
credible.  Specifically,  the  JSF  program  office  should 

•  include  costs  that  were  inappropriately  omitted  from  the  estimate; 

•  identify  performance  requirements  that  have  been  traded  off  in 
development; 

•  fully  document  assumptions,  data  sources  and  methodologies  in  the 
cost  model;  and 

•  perform  a  risk  and  uncertainty  analysis  to  focus  on  key  cost  drivers  and 
reduce  the  risk  of  cost  overruns. 

2.  The  program  conduct  a  full  Schedule  Risk  Analysis  to  ensure  that  its 
schedules  are  fully  understood,  manageable,  and  executable; 

3.  DOD  conduct  a  full,  independent  cost  estimate  should  be  conducted 
according  to  the  highest  standards  of  any  DOD  cost  estimating 
organization,  based  on  a  comprehensive  review  of  program  data;  that 
this  cost  estimate  be  reviewed  by  an  independent  third  party  such  as 
the  CAIG;  and  that  the  results  of  these  estimates  be  briefed  to  all 
interested  parties  in  DOD  and  Congress. 
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Agency  Comments 
and  Our  Response 


DOD  provided  us  with  written  comments  on  a  draft  of  this  report.  The 
comments  appear  in  appendix  III.  DOD  also  provided  several  technical 
comments,  which  we  incorporated  in  this  report. 

DOD  substantially  agreed  with  our  recommendation  to  revisit  the  Mid- 
Course  Risk  Reduction  plan.  DOD  stated  that  the  plan  is  a  cost-effective 
approach  with  a  manageable  level  of  risk  that  will  be  monitored  and 
revised  if  necessary.  We  believe  the  plan’s  reduction  of  test  resources  will 
hamper  development  testing  and  that  the  Department  will  eventually  have 
to  make  programmatic  adjustments,  adding  cost  and  time. 

DOD  also  substantially  agreed  with  our  three  recommendations  on  cost 
estimating.  DOD  indicated  that  it  will  implement  all  elements  except  the 
risk  and  uncertainty  analysis,  which  is  unwarranted  in  its  view.  We  believe 
that  risk  and  uncertainty  analysis  is  an  important  tool  that  establishes  a 
confidence  interval  for  a  range  of  possible  costs — as  opposed  to  a  single¬ 
point  estimate — and  facilitates  good  management  decisions  and  oversight. 
Such  analysis  is  a  best  practice  in  our  Cost  Assessment  Guide  and  we  note 
that  OSD’s  Cost  Analysis  Improvement  Group  supports  and  uses  this  cost¬ 
estimating  tool. 


We  are  sending  copies  of  this  report  to  the  Secretary  of  Defense;  the 
Secretaries  of  the  Air  Force,  Army,  and  Navy;  and  the  Director  of  the 
Office  of  Management  and  Budget.  We  will  also  provide  copies  to  others 
on  request.  In  addition,  the  report  will  be  made  available  at  no  charge  on 
the  GAO  Web  site  at  http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  concerning  this  report,  please 
contact  me  at  (202)  512-4841.  Contact  points  for  our  offices  of 
Congressional  Relations  and  Public  Affairs  may  be  found  on  the  last  page 
of  this  report.  Staff  members  making  key  contributions  to  this  report  are 
listed  in  appendix  IV. 


Michael  J.  Sullivan 
Director 

Acquisition  and  Sourcing  Management 
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List  of  Congressional  Committees 

The  Honorable  Carl  Levin 
Chairman 

The  Honorable  John  McCain 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Daniel  K.  Inouye 
Chairman 

The  Honorable  Ted  Stevens 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Ike  Skelton 
Chairman 

The  Honorable  Duncan  L.  Hunter 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  John  P.  Murtha 
Chairman 

The  Honorable  C.W.  Bill  Young 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 
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To  determine  the  Joint  Strike  Fighter  (JSF)  program’s  progress  in  meeting 
cost,  schedule,  and  performance  goals,  we  received  briefings  by  program 
and  contractor  officials  and  reviewed  financial  management  reports, 
budget  documents,  annual  Selected  Acquisition  Reports,  monthly  status 
reports,  performance  indicators,  and  other  data.  We  compared  reported 
progress  with  prior  years’  data,  identified  changes  in  cost  and  schedule, 
and  obtained  officials’  reasons  for  these  changes.  We  interviewed 
Department  of  Defense  (DOD),  JSF  program,  and  contractor  officials  to 
obtain  their  views  on  progress,  ongoing  concerns  and  actions  taken  to 
address  them,  and  future  plans  to  complete  JSF  development  and  ramp  up 
procurement. 

To  assess  plans  and  risks  in  development,  manufacturing,  and  test 
activities,  we  examined  program  documents  and  interviewed  DOD  and 
contractor  officials  about  changes  to  the  test  plan  and  actions  taken  to 
modify  these  plans  to  address  funding  and  schedule  challenges.  This 
included  reviewing  and  interviewing  program  and  Office  of  the  Secretary 
of  Defense  (OSD)  officials  about  changes  to  development  testing  that 
evolved  in  response  to  a  projected  shortfall  in  management  reserves  and  a 
goal  to  stay  on  schedule  toward  a  full-rate  production  decision  in  October 
2013.  We  reviewed  information  compiled  by  program  officials  to  document 
options  they  considered  viable,  changes  to  the  test  plan  and  test  resources 
that  would  occur  under  a  proposed  risk  reduction  option,  and 
challenges/risks  to  taking  this  course  of  action  and  possible  fallback  plans. 
We  also  reviewed  stakeholder  views  of  options  and  the  benefits  and 
challenges  of  going  forward  with  the  changes  made  to  the  development 
test  plan.  We  collected  manufacturing  cost  and  work  performance  data  to 
assess  progress  against  plans,  determined  reasons  for  manufacturing 
delays,  discussed  program  and  contractor  plans  to  improve,  and  expected 
impacts  on  development  and  operational  tests. 

In  assessing  program  cost  estimates,  we  also  evaluated  the  JSF  joint 
program  office  estimating  methodologies,  assumptions,  and  results  to 
determine  whether  the  official  cost  estimates  were  comprehensive, 
accurate,  well  documented,  and  credible.  We  used  our  draft  guide  on 
estimating  program  schedules  and  costs,  which  is  based  on  extensive 
research  of  best  practices.  Our  Cost  Assessment  Guide  considers  an 
estimate  to  be  accurate  if  it  is  not  overly  conservative,  is  based  on  an 
assessment  of  the  most  likely  costs,  and  is  adjusted  properly  for  inflation; 
comprehensive  if  its  level  of  detail  ensures  that  all  pertinent  costs  are 
included  and  no  costs  are  double-counted;  well  documented  if  the 
estimate  can  be  easily  repeated  or  updated  and  can  be  traced  to  original 
sources  through  auditing;  and  credible  if  the  estimate  has  been  cross- 
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checked  with  an  independent  cost  estimate  and  a  level  of  uncertainty 
associated  with  the  estimate  has  been  identified.  We  also  interviewed  the 
JSF  program  office’s  cost  estimating  team  to  obtain  a  detailed 
understanding  of  the  cost  model  and  met  with  the  Department  of  Defense 
Cost  Analysis  Improvement  Group  (CAIG) 1  to  understand  their 
methodology,  data  and  approach  in  developing  their  Joint  Strike  Fighter 
independent  cost  estimate.  We  analyzed  earned  value  management  (EVM)2 
reports  and  met  with  the  Naval  Air  Systems  Command  and  the  Defense 
Contract  Management  Agency  (DCMA)  to  discuss  the  EVM  data  and  to 
obtain  their  independent  cost  estimates  for  JSF  development  efforts.  To 
assess  the  validity  and  reliability  of  prime  contractors’  earned  value 
management  systems  and  reports,  we  analyzed  the  EVM  reports  and 
reviewed  audit  reports  prepared  by  the  DCMA. 

To  identify  future  challenges,  we  continued  discussions  with  DOD  and 
contractor  officials  on  forward-looking  plans  and  areas  of  emphasis.  We 
analyzed  budget  requirements  from  successive  plans  and  tracked 
contributing  factors  to  changes  in  budget.  We  collected  information  on 
commonality  assessments  among  the  three  variants  and  trends.  With  Navy 
and  Marine  Corps  officials,  we  discussed  future  plans  on  the  employment 
and  quantity  mix  of  aircraft  and  identified  differences  in  plans  and 
perspectives.  We  discussed  past  and  present  weight  growth  issues  with 
engineers  and  plans  for  controlling  future  growth. 

In  performing  our  work,  we  obtained  information  and  interviewed  officials 
from  the  JSF  Joint  Program  Office,  Arlington,  Virginia;  Aeronautical 
Systems  Center,  Wright-Patterson  Air  Force  Base,  Ohio;  Naval  Air  Systems 
Command,  Patuxent  River,  Maryland;  Defense  Contract  Management 
Agency,  Fort  Worth,  Texas;  and  Lockheed  Martin  Aeronautics,  Fort  Worth, 


1  The  CAIG  serves  as  the  principal  advisory  body  to  the  milestone  decision  authority  on  all 
matters  concerning  an  acquisition  program’s  life-cycle  cost,  and  is  given  general 
responsibilities  for  establishing  DOD  policy  guidance  on  a  number  of  matters  relating  to 
cost  estimating.  The  independent  CAIG  cost  estimate  is  designed  to  assess  the  program 
office  estimate  and  ensure  realistic  cost  estimates  are  considered. 

2  Earned  value  management  is  a  method  of  tracking  the  value  of  work  accomplished  in  a 
given  period  and  comparing  it  with  the  planned  value  of  work  scheduled  and  the  actual 
cost  of  work  accomplished.  Its  use  is  required  by  federal  regulations.  EVM  places  special 
emphasis  on  efficiently  and  effectively  executing  work  through  the  development  and 
operation  of  management  control  system  that  includes  people,  systematic  processes,  and 
innovative  tools  and  techniques.  The  intent  of  EVM  is  to  help  program  managers,  and  the 
teams  who  support  them,  be  successful  by  allowing  them  to  operate  productively  in  the 
high-risk  environments  surrounding  them. 


Page  35 


GAO-08-388  Joint  Strike  Fighter 


Appendix  I:  Scope  and  Methodology 


Texas.  We  also  met  and  obtained  data  from  the  following  OSD  offices  in 
Washington,  D.C.:  Director,  Operational  Test  and  Evaluation;  Program 
Analysis  and  Evaluation;  Cost  Analysis  Improvement  Group;  Portfolio 
Systems  Acquisition  (Air  Warfare);  and  Systems  and  Software  Engineering. 
We  conducted  this  performance  audit  from  June  2007  to  February  2008  in 
accordance  with  generally  accepted  government  auditing  standards.  Those 
standards  require  that  we  plan  and  perform  the  audit  to  obtain  sufficient, 
appropriate  evidence  to  provide  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives.  We  believe  that  the  evidence 
obtained  provides  a  reasonable  basis  for  our  findings  and  conclusions 
based  on  our  audit  objectives. 
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The  Current  JSF 
Program  Office 
Acquisition  Cost 
Estimate  Is  Not 
Reliable 


The  $299.8  billion  acquisition  cost  estimate  for  JSF  is  not  reliable  because  it  is 
not  sufficiently  comprehensive,  accurate,  documented,  or  credible.  Cost¬ 
estimating  organizations  throughout  the  federal  government  and  industry  use 
certain  key  practices  to  produce  sound  cost  estimates  that  are  comprehensive 
and  accurate  and  that  can  be  easily  traced,  replicated,  and  updated.  GAO’s  Cost 
Assessment  Guide  outlines  practices  that,  if  followed  correctly,  should  result  in 
high-quality,  reliable,  and  valid  cost  estimates  that  management  can  use  for 
making  informed  decisions.  We  assessed  the  methodology  used  by  the  JSF 
program  office  to  determine  its  development  cost  estimate  against  four  best 
practices  characteristics,  which  are  that  an  estimate  should  be  comprehensive, 
accurate,  well  documented,  and  credible.1  We  found  that  the  JSF  program 
office  has  not  followed  best  practices  for  developing  a  reliable  and  valid  life 
cycle  cost  estimate  because  it  did  not  include  certain  key  costs,  assumptions 
used  to  develop  the  estimate  are  overly  optimistic,  the  estimate  is  not  well 
documented,  and  no  analysis  has  been  done  to  state  the  confidence  and 
certainty  it  has  in  its  estimate.  As  a  result  of  these  weaknesses,  the  JSF 
program  acquisition  cost  estimate  is  not  reliable  for  decision  making. 


The  JSF  Cost  Estimate  Is  Estimates  are  comprehensive  when  they  contain  a  level  of  detail  that  ensures 

N Ot  Comprehensive  that  all  pertinent  costs  are  included  and  no  costs  are  double-counted.  It  is 

important  to  ensure  the  completeness,  consistency,  and  realism  of  the 
information  contained  in  the  cost  estimate.  Our  review  of  the  JSF  development 
cost  estimate  showed  that  there  are  several  cost  categories  totaling  more  than 
$10  billion  that  are  excluded  or  underreported  in  the  program  office  estimate. 
These  items  are  summarized  in  table  5. 


Table  5:  Costs  Excluded  from  the  Program  Office  Acquisition  Cost  Estimate 

Cost  item 

Possible  impact 

Alternate  engine 

$6.8  billion 

Military  construction 

$1 .5  billion  plus 

Tooling 

$2  billion-$3  billion  plus 

Capabilities  dropped  from  development  and  deferred 

Unknown 

Total 

$10  billion  plus 

Source:  GAO  analysis  of  DOD  data 

1  These  four  best  practice  criteria  for  a  reliable,  high-quality  cost  estimate  can  be  mapped 
to  12  steps  of  a  high-quality  cost  estimating  process  that  have  been  identified  by  GAO  in  the 
Cost  Assessment.  Guide.  Chapter  1  of  the  guide  describes  the  12  steps  and  how  they  map  to 
the  four  best  practice  characteristics. 
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•  Alternate  engine  program.  Congress  has  been  interested  in  DOD 
developing  a  second  source  for  the  JSF  engine  to  induce  competition 
and  to  reduce  operational  risks  in  the  future  should  the  sole  engine 
develop  problems  requiring  the  grounding  of  all  JSFs.  DOD  has  not 
wanted  to  pursue  this  second  engine  source  and  twice  removed 
funding  from  the  JSF  program  line.  In  2005,  DOD  deleted  a  total  of 
about  $7.2  billion  from  the  JSF’s  development  and  procurement 
accounts  for  the  alternate  engine.  In  2006,  it  reinserted  $340  million  for 
the  program,  reflecting  only  development  funding  in  the  future  years 
defense  program.  This  omits  about  $6.8  billion  left  out  of  the  JSF  cost 
estimate  for  this  program. 

•  Military  construction.  In  prior  years,  the  JSF  cost  estimate  included 
$2  billion  for  military  construction  costs.  Since  the  services  had  not  yet 
fully  established  basing  plans,  this  amount  was  a  top-level  parametric 
estimate  not  based  on  discrete  estimates  for  specific  sites.  The  current 
December  2006  cost  estimate  reported  military  construction  costs  of 
$533  million,  reflecting  only  the  amount  budgeted  in  the  fiscal  year 
2008  future  years  defense  program.  This  means  that  about  $1.5  billion 
in  military  construction — and  possibly  more — will  eventually  be 
required  for  specific  basing  needs  of  the  JSF  fleets.  DOD  will  update 
military  construction  estimates  as  the  services  identify  specific  site 
requirements. 

•  Tooling.  The  JSF  program  recently  increased  its  estimate  of  tooling 
costs  due  to  the  inclusion  of  additional  tooling  requirements  and 
estimating  methodology  changes.  This  change  is  ongoing,  and  has  not 
yet  been  included  in  official  program  estimates.  According  to  a  recent 
press  report,  a  Lockheed  Martin  official  stated  that  the  full  requirement 
to  support  procurement  by  our  allies  was  not  adequately  factored  into 
prior  tooling  estimates.  The  program  estimates  the  additional  cost 
through  2015  at  about  $2.1  billion. 

•  Deferred  capabilities.  Cost  and  performance  trade-offs  during 
development  have  resulted  in  some  requirements  being  deleted  from 
the  program  cost  estimate  and  deferred  until  later  years.  This  includes 
a  number  of  planned  capabilities  dropped  from  the  final  block  of 
development  software.  The  program  office  has  not  quantified  the  cost 
of  these  deferred  capabilities,  and  the  costs  are  not  reflected  anywhere 
in  the  program  office’s  life-cycle  cost  estimate.  Naval  Air  Systems 
Command  (NAVAIR)  officials  told  us  that  the  total  deferred  amount 
could  be  in  the  billions  of  dollars.  We  note  that  prior  acquisitions  such 
as  the  Global  Hawk  and  F-22A  programs  also  deferred  requirements 
that  would  later  need  additional  funding.  For  example,  we  reported  in 
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2005  that  the  Global  Hawk  program  costs  did  not  include  $400.6  million 
in  known  additional  procurement  costs  for  sensors,  ground  station 
enhancements,  and  other  items  required  to  achieve  the  system’s  initial 
full-up  capability. 2  These  costs  had  been  in  the  program  baseline  but 
were  later  deferred  and  reclassified  because  of  cost  pressures  and 
schedule  changes.  Similarly,  the  Air  Force’s  $5.9  billion  modernization 
and  reliability  improvement  program  includes  capabilities  deferred 
from  the  acquisition  program  and  reliability  enhancements  needed  to 
correct  deficiencies  and  achieve  the  level  of  reliability  that  was 
supposed  to  be  accomplished  during  acquisition. 


The  JSF  Cost  Estimate  Is  Estimates  are  accurate  when  they  are  based  on  an  assessment  of  the  costs 
Not  Accurate  most  likely  to  be  incurred.  Therefore,  when  costs  change,  best  practices 

require  that  the  estimate  be  updated  to  reflect  changes  in  technical  or 
program  assumptions  and  new  phases  or  milestones.  DOD’s  Cost  Analysis 
Improvement  Group  found  that  the  assumptions  the  JSF  program  office 
used  for  weight  growth,  staffing  head  counts,  commonality  savings  for 
cousin  (similar)  parts,  and  outsourced  labor  rate  savings  could  be  too 
optimistic,  given  the  program’s  complexity.  With  three  variants  and  three 
engines  (cruise,  alternate,  and  lift)  in  development,  multiple  customers, 
and  more  than  double  the  amount  of  operational  flight  software  than  the 
F-22A  and  four  times  that  of  the  F/A-18E/F,  the  JSF  acquisition  program  is 
substantially  more  complex  than  those  contemporary  systems,  and 
therefore  may  not  merit  assumptions  that  are  even  as  optimistic  as  the 
historical  data  for  those  programs.  The  following  table  shows  some  major 
differences  in  assumptions  used  by  the  program  office  and  the  CAIG  in 
estimating  JSF  costs. 


2  GAO,  Unmanned  Aircraft  Systems:  Global  Hawk  Cost  Increase  Understated  in  Nunn- 
McCurdy  Report,  GAO-06-222R  (Washington,  D.C.:  Dec.  15,  2005). 
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Table  6:  Major  Assumptions  in  JSF  Cost  Estimates 

Assumption 

JSF  Program  office 

CAIG 

Engineering  head  count 

5,000  engineers 

3,000  engineers 

Weight  growth 

3%  average  growth  based  on 
Lockheed  Martin  database 

6%  based  on  historical  data 

Cousin  parts  (similar,  but 
not  identical,  parts  among 
the  variants) 

82%  credit  for  commonality 
among  cousin  parts 

25%  credit  for  commonality 
among  cousin  parts 

Labor  cost  savings  for 
outsourcing 

50%  cost  savings 

None  -  based  labor  costs  on 
forward  pricing  rate 
agreements 

Additional  2,000  pounds 
for  carrier  variant 

No  cost  increase 

Estimated  a  cost  increase 

Fee,  and  “fee  on  fee”  for 
Northrop  Grumman  and 
BAE  Systems  production 
items 

13%  fee  on  all  development 
and  production  contracts 

No  “fee  on  fee”  impact 

15%  fee  on  all  development 
and  production  contracts 

Estimated  a  “fee  on  fee” 
impact 

Test  failure  redesign  effort 

No  additional  costs 

Additional  costs  for  this  effort 

Source:  JSF  program  office,  CAIG. 


JSF  program  officials  told  us  that  they  use  Lockheed  Martin  earned  value 
management  data  in  creating  their  estimate  of  JSF  development  costs. 
However,  DCMA,  which  reviews  contracts  and  industrial  performance  for 
DOD,  identified  this  data  as  being  of  very  poor  quality,  calling  into 
question  the  accuracy  of  any  estimate  based  on  these  data.  In  November 
2007,  DCMA  issued  a  report  saying  that  Lockheed  Martin’s  tracking  of  cost 
and  schedule  information  at  its  aerospace  unit  in  Fort  Worth,  Texas — 
where  the  JSF  program  is  managed — is  deficient  to  the  point  where  the 
government  is  not  obtaining  useful  program  performance  data  to  manage 
risks. 

DCMA  said  that  Lockheed’s  earned  value  data  at  the  Fort  Worth  facility 
are  not  sufficient  to  manage  complex,  multibillion-dollar  weapon  systems 
acquisition  programs.  Among  other  problem  areas,  DCMA  found  that 
Lockheed  had  not  clearly  defined  roles  and  responsibilities,  and  was  using 
management  reserve  funds  to  alter  its  own  and  subcontractor 
performance  levels  and  cost  overruns.  These  issues  hurt  DOD’s  ability  to 
use  the  Lockheed  data  to  determine  product  delivery  dates  and  develop 
accurate  estimates  of  program  costs.  DCMA  officials  who  conducted  the 
review  at  Lockheed  Martin  told  us  that  the  poor  quality  of  the  data 
invalidated  key  performance  metrics  regarding  cost  and  schedule,  as  well 
as  the  contractor’s  estimate  of  the  cost  to  complete  the  contract.  NAVAIR 
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had  also  raised  concerns  about  Lockheed  Martin’s  earned  value  system  as 
early  as  June  2005,  and  these  officials  told  us  they  were  in  agreement  with 
the  findings  in  the  November  2007  DCMA  report.  NAVAIR  officials  also 
said  that  most  deficiencies  identified  by  the  DCMA  report  have  the  effect 
of  underreporting  costs,  and  that  the  official  program  cost  estimates  will 
increase  if  the  deficiencies  are  corrected. 

Also  in  2007,  the  prime  contractor  alerted  DOD  to  a  billing  error  involving 
duplicate  charges  for  the  portion  of  the  earned  award  fee  paid  to 
subcontractors.  This  resulted  in  $266  million  in  overcharges.  Government 
officials  became  concerned  that  such  a  large  discrepancy  could  occur 
without  the  government’s  knowledge  and  questioned  the  adequacy  of  the 
contractor’s  billing  system  and  accounting  procedures.  DCMA  and  the 
Defense  Contract  Audit  Agency  were  tasked  to  conduct  an  investigation. 
Their  investigation  found  that  the  overbilling  resulted  from  an  accounting 
system  error  in  the  internal  handling  of  award  fees  on  the  JSF  contract. 
According  to  the  investigation  report,  the  error  that  created  the  overbilling 
has  been  corrected,  and  the  government  has  recouped  the  overbilled 
principal  and  interest. 


The  JSF  Cost  Estimate  Is  Cost  estimates  are  well  documented  when  they  can  be  easily  repeated  or 
Not  Well  Documented  updated  and  can  be  traced  to  original  sources  through  auditing.  Rigorous 

documentation  increases  the  credibility  of  an  estimate  and  helps  support 
an  organization’s  decision-making  process.  The  documentation  should 
explicitly  identify  the  primary  methods,  calculations,  results,  rationales  or 
assumptions,  and  sources  of  the  data  used  to  generate  each  cost  element. 
All  the  steps  involved  in  developing  the  estimate  should  be  documented  so 
that  a  cost  analyst  unfamiliar  with  the  program  can  recreate  the  estimate 
with  the  same  result. 

We  found  that  the  JSF  cost  model  is  highly  complex  and  the  level  of 
documentation  is  not  sufficient  for  someone  unfamiliar  with  the  program 
to  easily  recreate  it.  Specifically,  we  found  that  the  program  office  does 
not  have  formal  documentation  for  the  development,  production,  and 
operating  support  cost  models.  Instead,  it  relies  on  briefing  slides  that 
describe  the  methodology  and  the  data  sources  used,  but  did  not  provide 
detailed  documentation  such  as  quantitative  analysis  to  support  the 
assumptions  that  were  involved  in  producing  the  life-cycle  cost  estimate. 
For  the  development  cost  estimate,  the  JSF  program  office  admitted  it  did 
not  have  a  cost  model  that  continually  updates  with  actual  costs.  Instead 
the  program  office  relies  heavily  on  earned  value  management  data  and 
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analysis  from  Lockheed  Martin  to  update  its  development  cost  estimate, 
but  provided  us  no  documentation  to  back  up  this  claim. 


The  JSF  Cost  Estimate  Is  Estimates  are  credible  when  they  have  been  cross-checked  with  an 
Not  Credible  independent  cost  estimate  and  when  a  level  of  uncertainty  associated  with 

the  estimate  has  been  identified.  An  independent  cost  estimate  provides 
the  estimator  with  an  unbiased  test  of  the  reasonableness  of  the  estimate 
and  reduces  the  cost  risk  associated  with  the  project  by  demonstrating 
that  alternative  methods  generate  similar  results. 

Several  independent  organizations  have  reviewed  the  JSF  program  and  are 
predicting  much  higher  costs  than  the  program  office.  Table  7  below 
provides  a  summary  of  these  assessments. 


Table  7:  Outside  Organizations’  Assessments  of  JSF  Cost  and  Schedule 

Assessing 

organization 

Impact  on  cost 

Impact  on  schedule 

CAIG 

$5.1  billion  more  for  development,  over 
$33  billion  more  for  procurement 

12  months  slip 

NAVAIR 

$8  billion  to  $13  billion  additional 
development  costs  or  trade-offs  adding 
to  cost  in  procurement 

19-27  months  slip 

DCMA 

$4.9  billion  additional  cost  to  complete 
Lockheed  Martin  contract,  including  the 
cost  of  a  12-month  schedule  slip 

Up  to  12  months  slip 

Source:  CAIG,  NAVAIR,  DCMA. 


In  2005,  the  CAIG  performed  an  independent  estimate  of  JSF  program 
development  costs,  which  include  the  cost  of  the  Lockheed  Martin 
contract  and  fees  as  well  as  the  government’s  in-house  costs.  The  CAIG 
expected  that  the  development  phase  would  cost  $5. 1  billion  more  than 
expected  by  the  program  office,  measured  against  the  program  office’s 
most  recent  data  available  at  that  time,  from  the  2004  Selected  Acquisition 
Report  (SAR).  The  CAIG  official  in  charge  of  the  estimate  told  us  that 
while  it  has  not  formally  presented  an  updated  estimate  to  the  program 
office,  the  order  of  magnitude  of  difference  between  the  CAIG  and 
program  office  estimates  remains  roughly  the  same  as  at  the  time  of  the 
2005  independent  estimate. 


Page  42 


GAO-08-388  Joint  Strike  Fighter 


Appendix  II:  GAO  Assessment  of  JSF  Program 
Cost  Estimate 


Table  8:  JSF  Development  Program  Cost  Estimate  Comparison 

(Then-year  dollars  in  billions) 

JSF 

CAIG 

Delta  between 
CAIG  And  JSF 

System  development  start  in  2001 

$30.2 

$31.4 

$1.2 

December  2004  SAR 

$41.5 

$46.6 

$5.1 

Cost  growth  from  2001  start  to  2004  SAR 

$11.3 

$15.2 

$3.9 

Source:  CAIG. 


The  variance  between  the  CAIG  and  program  office  estimates  grew 
significantly  as  the  program  encountered  problems.  The  CAIG  explained 
that  some  of  the  $15.2  billion  growth  in  its  development  cost  estimate  from 
Milestone  B  in  2001  to  the  December  2004  SAR  was  due  to  initial 
assumptions  that  5,000  engineers  would  be  available  to  work  on  the  three 
JSF  variants.  This  assumption  turned  out  to  be  too  optimistic  since  only 
about  3,000  engineers  have  been  working  on  the  program.  Because  of 
fewer  people  available  to  support  the  JSF  design  and  development,  the 
CAIG  shifted  the  program  schedule  to  the  right,  increasing  the  costs.  The 
program  office,  on  the  other  hand,  assumed  it  could  get  the  same  effort 
done  with  fewer  people.  In  addition,  the  CAIG  used  historical  data  from 
the  F-22A  program,  including  the  costs  to  design  the  aircraft,  test  it,  and 
redesign  any  fixes,  and  adjusted  these  data  to  account  for  differences  in 
the  JSF  program,  including  the  three  variants.  The  program  office  relies 
mostly  on  contractor  data. 

When  it  was  awarded  the  development  contract  in  2001,  Lockheed  Martin 
agreed  to  develop  the  JSF  aircraft  for  $16.5  billion,  excluding  fee.  In  April 
2005,  the  development  program  was  rebaselined,  adding  more  than  $6 
billion  to  reflect  funds  added  to  the  program  due  to  weight  growth  issues 
in  2003.  This  raised  the  JSF  baseline  development  contract  cost  estimate 
to  $23.2  billion,  excluding  fee. 

Despite  the  additional  funding  to  cover  preexisting  cost  and  schedule 
overruns,  Lockheed  Martin’s  JSF  development  cost  and  schedule 
performance  has  continued  to  decline  over  time.  As  shown  in  figure  6, 
cost  and  schedule  variances  continued  on  a  downward  trend  despite  the 
April  2005  rebaseline.  As  of  September  2007,  Lockheed  Martin  was 
reporting  cumulative  cost  overruns  of  $305.7  million  and  was  behind 
schedule  to  an  extent  valued  at  $251.3  million. 
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Figure  6:  Cost  and  Schedule  Variances  on  the  Aircraft  Development  Contract 


Variance  from  plan  (dollars  in  million) 
400 

200 


-1000 


Fiscal  year 


Cost  variance 

—  —  Schedule  variance 
Source:  GAO  analysis  of  DOD  data. 


Key  drivers  of  cost  overruns  to  date  have  included  unfunded  requirements 
for  design  changes,  loss  of  commonality  savings,  critical  part  shortages, 
high  change  traffic,  inefficient  productivity  due  to  performing  work  out  of 
sequence,  constant  rework,  suppliers’  performance,  late  release  of 
engineering  requirements,  a  greater  than  planned  effort  for  designs  of  the 
short  takeoff  and  landing  and  the  conventional  takeoff  and  landing 
variants,  and  additional  radar  testing.  Some  of  this  cost  variance  is  due  to 
optimistic  assumptions  at  the  beginning  of  the  program.  For  example, 
cost  estimates  assumed  that  only  one  design  iteration  would  be  needed, 
whereas  in  reality  it  takes  numerous  design  iterations  before  the  final 
designs  are  determined. 

Despite  its  poor  performance  since  the  rebaseline,  Lockheed  Martin  was 
predicting  only  a  $113  million  cost  overrun  at  contract  completion.  This  is 
unrealistic  given  the  persistence  and  size  of  the  $305.7  million  overrun 
reported  in  September  2007,  at  which  point  the  contract  was  67  percent 
complete.  In  order  to  achieve  a  $113  million  overrun  at  completion, 
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Lockheed  Martin  would  have  to  not  only  incur  no  further  cost  variances 
from  now  until  completion  of  the  contract,  but  it  would  also  have  to 
significantly  improve  its  performance.  This  is  unlikely  given  that  studies 
of  more  than  700  defense  programs  have  shown  limited  opportunity  for 
getting  a  program  back  on  track  once  it  is  more  than  15  percent  to  20 
percent  complete.  The  true  cost  to  complete  the  contract  may  be 
significantly  greater,  as  DCMA  has  expressed  its  concern  to  DOD  over 
Lockheed  Martin’s  failure  to  regularly  update  its  estimate  of  the  costs  to 
complete  the  JSF  contract,  stating  that  Lockheed’s  infrequent  updates  are 
insufficient  to  provide  the  government  with  information  bearing  on 
potential  cost  growth  and  funding  needs. 

Like  the  CAIG,  both  DCMA  and  NAVAIR  believe  that  Lockheed  Martin’s 
estimate  at  completion  is  too  optimistic  and  that  the  program  office  will 
most  likely  require  significantly  more  funding  to  complete  the 
development  program.  NAVAIR  provides  resources  to  the  JSF  program 
office  cost-estimating  function,  and  it  estimated  in  2006  that  JSF 
development  costs  could  be  almost  $8  billion  to  $13  billion  higher  than 
estimated  by  the  program  office,  or  else  cost  billions  more  in  procurement 
due  to  requirements  pushed  off  from  development.  NAVAIR  officials  told 
us  they  believe  that  the  2006  estimate  continues  to  be  accurate  today,  but 
explained  that  since  the  JSF  program  is  a  joint  program  they  do  not 
control  JSF  cost-estimating  procedures,  although  their  estimates  are 
briefed  to  JSF  program  management.  The  estimate  removed  what  NAVAIR 
views  as  artificial  constraints  on  the  JSF  schedule  and  projected  forward, 
resulting  in  an  estimate  that  the  schedule  would  likely  slip  19  to  27 
months,  and  combined  this  with  trends  in  cost  performance.  NAVAIR 
officials  said  that  their  confidence  in  the  achievability  of  the  JSF  program 
schedule  is  low,  as  the  master  schedule  comprises  more  than  600 
individual  schedules,  making  it  difficult  to  accurately  assess  the 
achievability  of  the  overall  schedule. 

DCMA  estimates  that  JSF  development  could  cost  as  much  as  $4.9  billion 
more  than  program  office  estimates,  accounting  for  poor  cost  and 
schedule  performance  to  date  and  assuming  further  schedule  slips  of  up  to 
12  months.  DCMA  confirmed  that  a  schedule  risk  analysis,  which  uses 
statistical  techniques  to  obtain  a  measure  of  confidence  in  completing  a 
program,  has  never  been  performed  on  the  JSF  program.  Since  historically 
state-of-the-art  development  programs  have  taken  longer  than  planned,  a 
schedule  risk  analysis  should  be  conducted  to  determine  the  level  of 
uncertainty  in  the  schedule.  Despite  these  outside  organizations’ 
predictions  of  significantly  higher  costs  to  complete  the  JSF  contract  and 
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the  lack  of  realism  in  the  contractor’s  own  estimate,  the  JSF  program 
office  continues  to  use  the  contractor’s  estimate  as  its  own. 


Table  9:  NAVAIR  and  DCMA  Estimates  at  Completion  for  JSF  Aircraft  Development 
Contract  (Then-year  dollars  in  billions) 

Estimates 

Projected 
overrun  to  the 
estimate 

Lockheed  Martin’s  estimate  at  completion  (EAC) 

$23.4 

NAVAIR  best  case  EAC  projection 

$31.3 

$7.9 

NAVAIR  worst  case  EAC  projection 

$36.8 

$13.4 

DCMA  EAC  based  on  earned  value  data 

$24.8 

DCMA  additional  costs  for  schedule  slip 

$3.6 

Total  DCMA  EAC  (earned  value  +  schedule  slip) 

$28.3 

$4.9 

Source:  GAO  analysis  of  DOD  data. 


In  addition  to  expected  cost  overruns  for  JSF  development,  the  CAIG  is 
predicting  significantly  higher  costs  for  JSF  for  the  military  services  to 
purchase  the  aircraft.  Using  different  assumptions  about  weight  growth, 
labor  rates,  avionics  and  propulsion  costs,  and  contractor  fees,  the  CAIG 
calculated  significantly  higher  unit  costs  for  the  aircraft  variants  (see  table 
6  earlier  in  this  report  for  comparison  of  CAIG  and  program  office 
assumptions  affecting  both  development  and  procurement  costs 
estimates).  Multiplying  these  higher  unit  costs  by  the  expected 
procurement  quantities  leads  to  a  more  than  $33  billion  (in  constant  year 
2002  dollars)  difference  from  official  program  office  estimates  for 
procurement  costs.3  The  CAIG  estimates  were  briefed  in  2006  to  the  DOD 
working  group  that  oversees  the  JSF  program,  and  top  OSD  officials  were 
aware  of  the  discrepancy  between  the  CAIG  and  JSF  program  office 
estimates. 

The  program  office  has  not  conducted  an  uncertainty  analysis  on  its  cost 
estimates  despite  the  complexity  of  the  program  and  associated  risk  and 
uncertainty.  As  shown  in  table  10,  the  JSF  program  is  significantly  more 
complicated  than  comparable  aircraft  development  programs. 


3  We  expressed  the  costs  in  constant  2002  dollars  instead  of  then-year  dollars  because  the 
data  we  relied  on  to  make  this  projection  were  in  constant  dollars,  and  we  did  not  have  the 
quantity  profiles  by  year  to  inflate  the  costs. 
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Table  10:  JSF  Complexity  Compared  to  That  of  Similar  Aircraft  Programs 

Complexity  factor 

F/A-18E/F 

F/A-22 

JSF 

Program  participation  by  multiple 
military  services 

No 

No 

Yes 

Aircraft  variants 

One 

One 

Three 

Avionics 

Off-the-shelf 

New 

New 

Stealth 

Minimal 

Yes 

Yes 

Software  (operational  flight  program 
source  lines  of  code) 

1.1  million 

2.2  million 

5.0  million8 

Engine(s) 

One 

One 

Two  cruise,  one  lift 

International  participation 

No 

No 

Yes 

Source:  CAIG 

aThis  assumes  approximately  30  percent  growth  in  lines  of  code  by  completion  of  development  (F-22 
included  34  percent  growth  and  F/A-18,  60  percent  growth). 


This  complexity  makes  it  all  the  more  necessary  to  fully  account  for  the 
effect  various  risks  can  have  on  the  overall  cost  estimate.  An  uncertainty 
analysis  assesses  the  extent  to  which  the  variability  of  an  outcome 
variable  is  caused  by  uncertainty  in  the  input  parameters.  It  should  be 
performed  for  every  cost  estimate  in  order  to  inform  decision  makers 
about  the  likelihood  of  success.  In  performing  uncertainty  analysis,  an 
organization  varies  the  effects  of  multiple  elements  on  costs,  and  as  a 
result,  can  express  a  level  of  confidence  in  the  point  estimate. 

We  found  that  the  JSF  program  has  not  conducted  an  uncertainty  analysis. 
Such  analysis  would  provide  a  range  of  possible  values  to  program 
management  and  an  estimate  of  the  likelihood  of  the  various  possibilities. 
Instead,  the  program  office  only  offers  a  single  point  estimate — one  dollar 
figure,  with  no  associated  range — and  no  technical  analysis  of  the 
likelihood  that  this  estimate  is  credible.  The  lead  cost  estimator  for  the 
program  office  acknowledged  that  such  a  single  point  estimate  is  virtually 
certain  to  be  wrong,  but  also  stated  that  the  analysis  used  to  develop  a 
range  of  values  is  easily  manipulated  and  therefore  not  valuable.  It  is 
GAO’s  view  that  a  point  estimate  should  be  accompanied  by  an  estimated 
confidence  level  to  quantify  the  uncertainty  surrounding  the  estimate  in 
order  for  management  to  make  good  decisions.  Because  the  JSF  program 
office  has  not  conducted  an  uncertainty  analysis,  it  is  unable  to  provide 
Congress  with  any  confidence  level  for  its  point  estimate  of  approximately 
$300  billion  for  JSF  acquisition. 
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OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 
3000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3000 

MAR  1  0  2088 


Mr.  Michael  J.  Sullivan 

Director,  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  N.W. 

Washington,  DC  20548 

Dear  Mr.  Sullivan: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  draft  report, 
“JOINT  STRIKE  FIGHTER:  Recent  Decisions  by  DoD  Add  to  Program  Risks,”  dated 
February  5,  2008  (GAO  Code  120663/GAO  08-388). 

The  Department  partially-concurs  with  the  report’s  first  two  recommendations  and 
concurs  with  the  final  two  recommendations.  The  Department  appreciates  the  GAO’s 
highlighting  of  both  program  successes  and  challenges.  Details  of  the  Department’s 
responses  are  contained  in  the  enclosure. 

The  Department  values  the  opportunity  to  comment  on  the  draft  report  and  looks 
forward  to  continued  discussions  on  the  Joint  Strike  Fighter  program  with  the  GAO  for 
next  year’s  audit. 


Sincerely, 


Portfolio  Systems  Acquisition 

Enclosure: 

As  stated 


o 


ACQUISITION, 
TECHNOLOGY 
AND  LOGISTICS 
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GAO  DRAFT  REPORT  DATED  FEBRUARY  5,  2008 
GAO-08-388  (GAO  CODE  120663) 


“JOINT  STRIKE  FIGHTER:  RECENT  DECISIONS  BY  DOD  ADD  TO 
PROGRAM  RISKS” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATIONS 


RECOMMENDATION  1:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
elements  of  the  Department  to  revisit  and,  if  appropriate,  revise  the  Mid-Course  Risk 
Reduction  plan  recently  approved.  (Page  3 1  /GAO  Draft  Report) 

DOD  RESPONSE:  Partially-Concur.  The  Department  continues  to  believe  the  Mid- 
Course  Risk  Reduction  (MCRR)  plan  is  a  cost  effective  approach  with  a  manageable 
level  of  risk.  USD(AT&L)  approved  the  plan  in  October  2007  after  thorough  review  of 
the  benefits  and  risks.  Implementation  of  the  plan  is  well  underway.  The  Department 
developed  metrics  to  monitor  and  evaluate  contractor  Management  Reserve 
replenishment  and  use,  manufacturing  line  progress,  and  the  MCRR  impacts  on 
developmental  testing.  The  plan  has  always  been  to  revisit  and,  if  necessary,  revise 
MCRR  implementation  if  metrics/monitoring  fail  to  achieve  expectations.  The 
Department  does  not  plan  to  alter  its  current  plan  of  monitoring  MCRR  execution. 

RECOMMENDATION  2:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
that  the  Joint  Strike  Fighter  (JSF)  program  office  update  its  cost  estimate  using  best 
practices,  so  that  the  estimate  is  comprehensive,  accurate,  well-documented,  and  credible. 
Specifically,  the  JSF  program  office  should: 

(a)  Include  costs  that  were  inappropriately  omitted  from  the  estimate; 

(b)  Identify  performance  requirements  that  have  been  traded  off  in  development; 

(c)  Fully  document  assumptions,  data  sources,  and  methodologies  in  the  cost 
model;  and 

(d)  Perform  a  risk  and  uncertainty  analysis  to  focus  on  key  cost  drivers  and  reduce 
the  risk  of  cost  overruns.  (Page  3 1/GAO  Draft  Report) 

DOD  RESPONSE:  Partially-Concur.  The  F-35  Program  Office  is  completing  a 
comprehensive  cost  estimate  in  preparation  of  an  independent  assessment  directed  by  the 
Department  of  Defense,  as  indicated  in  the  response  to  Recommendation  4.  The 
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Department  believes  that  a  Risk  and  Uncertainty  Analysis  as  defined  in  GAO-07-1 134SP 
Cost  Assessment  Guide  is  unwarranted  due  to  the  subjective  nature  of  the  modeling 
inputs. 

RECOMMENDATION  3:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
that  the  program  conduct  a  full  Schedule  Risk  Analysis  to  ensure  that  its  schedules  are 
fully  understood,  manageable,  and  executable.  (Page  31 /GAO  Draft  Report) 

POD  RESPONSE:  Concur.  A  combined  Navy,  Air  Force,  and  OSD  team  will  consider 
schedule  risk  in  a  planned  joint  cost  estimate  update,  discussed  in  more  detail  in  the 
response  to  Recommendation  4. 

RECOMMENDATION  4:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
that  a  full,  independent  cost  estimate  be  conducted  according  to  the  highest  standards  of 
any  DoD  cost  estimating  organization,  based  on  a  comprehensive  review  of  program 
data;  that  this  cost  estimate  be  reviewed  by  an  independent  third  party  such  as  the  Cost 
Analysis  Improvement  Group;  and  that  the  results  of  these  estimates  be  briefed  to  all 
interested  parties  in  DoD  and  the  Congress.  (Page  31/GAO  Draft  Report) 

DOD  RESPONSE:  Concur.  In  January  2008,  the  Department  directed  an  independent 
joint  cost  assessment  to  be  performed  by  a  team  of  Navy,  Air  Force,  and  OSD  cost 
experts.  This  joint  cost  assessment  was  initiated  at  the  request  of  the  F-35  Program 
Executive  Office  (PEO)  and  will  aid  the  Department’s  development  of  the  FY  2010 
President’s  Budget  request. 

The  Department  is  pleased  that  the  GAO  endorses  a  path  the  PEO  and  OSD  have  already 
developed,  i.e.,  a  joint  independent  F-35  cost  review.  The  Air  Force  Cost  Analysis 
Agency  (AFCAA),  NAVAIR  4.2,  and  OSD  Cost  Analysis  Improvement  Group  (CAIG) 
are  participating  in  the  review.  We  appreciate  the  fact  that  GAO  recognizes  the 
complexities  involved  in  estimating  F-35  costs. 

FACTUAL  DISCREPANCIES:  There  are  three  significant  discrepancies  noted  within 
the  report.  The  March  2004  re-plan  added  $7.5  billion  to  the  development  estimate,  not 
“over  10  billion”  dollars  as  indicated  in  the  report.  Additionally,  the  December  2006  cost 
estimate  reflected  an  increase  in  the  Military  Construction  (MILCON),  not  a  reduction  as 
GAO  states.  The  past  six  annual  Selected  Acquisition  Reports  assert  that  MILCON 
estimates  will  continue  to  be  updated  as  the  Services  identify  specific  site  requirements. 
Lastly,  GAO  cites  a  total  procurement  cost  increase  of  28%  since  March  2004  but  omits 
the  fact  that  the  increase  is  less  than  half  of  that  (i.e.,  13%)  when  inflation  impacts  are 
excluded. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and  policies; 
and  provides  analyses,  recommendations,  and  other  assistance  to  help 
Congress  make  informed  oversight,  policy,  and  funding  decisions.  GAO’s 
commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost 
is  through  GAO’s  Web  site  (www.gao.gov).  Each  weekday,  GAO  posts 
newly  released  reports,  testimony,  and  correspondence  on  its  Web  site.  To 
have  GAO  e-mail  you  a  list  of  newly  posted  products  every  afternoon,  go 
to  www.gao.gov  and  select  “E-mail  Updates.” 
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more  copies  mailed  to  a  single  address  are  discounted  25  percent.  Orders 
should  be  sent  to: 
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441  G  Street  NW,  Room  LM 

Washington,  DC  20548 

To  order  by  Phone:  Voice:  (202)  512-6000 

TDD:  (202)  512-2537 

Fax:  (202)  512-6061 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 
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Ralph  Dawn,  Managing  Director,  dawnr@gao.gov,  (202)  512-4400 
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